Celery with Django: Asynchronous Task Processing 🎯

Welcome back to CodeYourCraft! Today, we're diving into an exciting topic - Celery with Django. This powerful combination allows us to process tasks asynchronously, making our applications more efficient and responsive. Let's get started!

What is Celery? 📝

Celery is a distributed task queue based on the message passing paradigm. It is used to offload time-consuming tasks from a web application, allowing for better performance and scalability. Celery can be integrated seamlessly with Django to create robust, asynchronous applications.

Why Use Celery with Django? 💡

Improve application responsiveness: By offloading tasks to Celery, the main application remains lightweight and quick, providing a better user experience.
Scale easily: Celery can handle a large number of tasks concurrently, making it easy to scale applications as the workload increases.
Flexible task management: Celery provides various strategies for task management, such as retrying failed tasks, delaying task execution, and more.

Setting Up Celery with Django 💡

Prerequisites

Python 3.x
Django 3.x
Celery 5.x

Installation

Install Celery using pip:

bash

pip install celery

Install Redis (a popular message broker for Celery) using pip:

bash

pip install redis

Creating a Celery Application

Create a new Django app:

bash

python manage.py startapp celery_app

Create a new Celery app in celery_app/ celery.py:

python

from celery import Celery

app = Celery('celery_app')

app.conf.beat_schedule = {
    'run-every-minutes': {
        'task': 'celery_app.tasks.example_task',
        'schedule': 60.0,
    },
}

if __name__ == "__main__":
    app.start()

Define the example task in celery_app/tasks.py:

python

from celery import shared_task

@shared_task
def example_task():
    print("Example task is running...")

Running Celery

Add the celery app to the installed apps in the celery_app/__init__.py file:

python

INSTALLED_APPS = [
    ...
    'celery_app',
    ...
]

Include the celery app in your Django project's urls.py file:

python

from django.urls import path, include

urlpatterns = [
    ...
    path('admin/', admin.site.urls),
    path('api/', include('celery_app.urls')),
    ...
]

Add the following to your project's main settings.py file:

python

BROKER_URL = 'redis://localhost:6379/0'
CELERY_RESULT_BACKEND = 'redis://localhost:6379/0'

Run Celery using the command:

bash

celery -A celery_app worker --loglevel=info

Running the Example Task ✅

Now that Celery is set up, let's run the example task:

Create a new view in celery_app/views.py:

python

from django.http import HttpResponse
from celery_app.tasks import example_task

def run_example_task(request):
    example_task.apply_async(countdown=10)
    return HttpResponse("Example task scheduled to run in 10 seconds.")

Add the new view to the app's urls.py file:

python

from django.urls import path
from . import views

urlpatterns = [
    path('run_task/', views.run_example_task, name='run_task'),
    ...
]

Now, if you navigate to http://localhost:8000/run_task/, you'll see the message "Example task scheduled to run in 10 seconds." After 10 seconds, the example task will run in the background.

Celery Tasks and Results 📝

In Celery, tasks are functions that can be executed asynchronously. Results of these tasks can be retrieved, allowing you to handle the results in your application.

Sending a Task

python

result = example_task.apply_async(args=[1, 2, 3])

Retrieving a Task Result

python

result.get(timeout=60)

That's it for today! You've learned how to set up and use Celery with Django. In the next lesson, we'll delve deeper into Celery tasks, task scheduling, and task results.

Quiz 📝

Happy learning, and see you in the next lesson! 🚀