KEMBAR78
Introduction to Django-Celery and Supervisor | PPT
Introducti
on to
Celery
What is Celery?
“Celery is an asynchronous task queue
based on distributed message
passing.”
It is a powerful, production-ready
asynchronous job queue, which allows
you to run time-consuming Python
functions in the background.
Why should I use
Celery?
User perspective
• Minimize request/response cycle
• Smoother user experience
• Difference between pleasant and unpleasant
Developer perspective
• Offload time/cpu intensive processes
• Scalability - add workers as needed
• Flexibility - many points of customization
• Actively developed
• Great documentation
• Lots of tutorials
OUT OF THE
REQUEST/RESPONSE
CYCLE
• Example: Sending emails asynchronously.
TASKS IN THE
BACKGROUND.
• Example: Computational heavy jobs.
• Example: Interacting with external APIs.
PERIODIC JOBS
Celery Architecture
PRODUCER
• Produces a task for the queue.
BROKER
• Stores the task backlog
• Answers, what work remains to be done?
• RabbitMQ, Redis, SQLAlchemy, Django's
ORM, MongoDB
WORKER
• Execute and consumes tasks.
• Distributed.
RESULTS BACKEND
• Stores the results from our tasks.
EXAMPLE
INTERGRATING WITH
DJANGO.
BEWARE OF DJANGO-
CELERY
IMPORTANT SETTINGS
• BROKER_URL = 'redis://localhost:6379/0'
• CELERY_RESULT_BACKEND = 'redis://localhost:6379/0'
• CELERY_TASK_RESULT_EXPIRES = 7*86400 # 7 days
• CELERYBEAT_SCHEDULER =
"djcelery.schedulers.DatabaseScheduler"
• CELERY_ACCEPT_CONTENT = ['application/json']
• CELERY_TASK_SERIALIZER = 'json'
• CELERY_RESULT_SERIALIZER = 'json'
• BROKER_BACKEND = "redis"
• BROKER_CONNECTION_TIMEOUT = 5.0
• CELERY_IMPORTS = ('<your-app-name>.tasks',)
Write code in celery.py
Intialize celery in __init__.py
BEST PRACTICES
NEVER PASS OBJECTS AS
ARGUMENTS.
AVOID LAUNCHING SYNCHRONOUS
SUBTASKS
PERIODIC TASKS
THINGS GO WRONG ?
RE-TRY!
Thanks!

Introduction to Django-Celery and Supervisor