Django Cellery stuck as Pending
I am attempting to use celery within my django app to speed up processing time of a function and I can't get it to work correctly. I am using RabbitMQ.
my tasks.py
from celery import Celery
from celery import shared_task,current_task
from myapp.celery import app
@app.task
def add(x,y):
for i in range(25000000):
a = x+y
return x+y
my python code
def test_multi_func():
x = 5
y = 10
i = 15
print(f"start = {time.perf_counter()}")
while i > 0:
g = add.delay(x,y)
result = add.AsyncResult(g)
i -= 1
print(result.backend)
print(result.status)
print(f"end = {time.perf_counter()}")
print(f"g = {g.get()}")
print(f"x = {x} ; y = {y}")
my settings.py
INSTALLED_APPS = [
'django.contrib.admin',
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.messages',
'django.contrib.staticfiles',
'django_select2',
'chartjs',
'django_material_icons',
'material',
'django_celery_results',
'celery_progress',
'django_apscheduler'
]
BROKER_URL = 'django://'
result_backend = 'django-db'
CELERY_RESULT_BACKEND = 'django-db'
result_persistent = True
task_result_expires = None
send_events = True
CELERY_ACCEPT_CONTENT = ['json']
CELERY_TASK_SERIALIZER = 'json'
CELERY_RESULT_SERIALIZER = 'json'
CELERY_BROKER_URL = 'amqp://localhost'
CELERY_CACHE_BACKEND = 'django-cache'
CELERY_IGNORE_RESULT = False
my celery.py
import os
from celery import Celery
# Set the default Django settings module for the 'celery' program.
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'myapp.settings')
app = Celery('myapp',
backend='amqp://guest@localhost:15672/',
broker='amqp://guest@localhost:15672/',
include=['myapp.tasks'])
app.config_from_object('django.conf:settings', namespace='CELERY')
app.autodiscover_tasks()
@app.task(bind=True)
def debug_task(self):
print(f'Request: {self.request!r}')
What am I doing wrong? I am have a difficult time understanding this. It triggers the test_multi_func() and then in stuck as pending within RabbitMQ and nothing happens. Would really appreciate if someone could help me understand what I need to do differently. I've tried many different iterations of different code I could find online and nothing seems to work.
Your celery configuration has many conflicts, and I'm wondering how you're running the worker. Specifically,
In your
settings.py,
you're mixing the legacy and new styles of configuring Celery. The lowercased configuration settings are preferred for Celery 4 and beyond: https://docs.celeryq.dev/en/stable/userguide/configuration.html#new-lowercase-settingsYou've configured the broker URLs twice (
amqp://guest@localhost:15672/
andamqp://localhost
), and you've configured two different result backends.You've passed in the configuration to the Celery instance three times, and the settings are stomping on each other. Specifically, when we pass your settings directly into the Celery class is should look like the following:
app = Celery('myapp',
backend_url='amqp://guest@localhost:15672/',
broker_backend='amqp://guest@localhost:15672/',
include=['myapp.tasks'])
You can also create the Celery instance and configure it in two steps if you prefer:
app = Celery('myapp')
app.conf.update(broker_url='amqp://guest@localhost:15672/',
result_backend='amqp://guest@localhost:15672/')