Celery Problem - Docker unable to run Celery background task, Celery does not seem to start properly - Docker, Django, React, Redis, Celery

I have created a test repository that has Django, React, Redis, Celery: https://github.com/axilaris/docker-django-react-celery-redis. My goal is to get celery working but its not.

This is based from this Docker - React and Django example tutorial code: https://github.com/dotja/authentication_app_react_django_rest

And trying to use Docker - Celery & Redis from this tutorial code: https://github.com/veryacademy/docker-mastery-with-django/tree/main/Part-4%20Django%20Postgres%20Redis%20and%20Celery <-- Part 4 tutorial for Celery & Redis

I am having trouble to get Celery started correctly as I see the background task is not executed.

It did not print this. I think this is the problem.

celery       |  -------------- celery@b755a7cdba8d v5.3.6 (emerald-rush)
celery       | --- ***** ----- 
celery       | -- ******* ---- Linux-6.6.12-linuxkit-aarch64-with 2024-03-02 20:48:06
celery       | - *** --- * --- 
celery       | - ** ---------- [config]
celery       | - ** ---------- .> app:         core:0xffff9bbd7550
celery       | - ** ---------- .> transport:   redis://redis:6379//
celery       | - ** ---------- .> results:     redis://redis:6379/
celery       | - *** --- * --- .> concurrency: 10 (prefork)
celery       | -- ******* ---- .> task events: OFF (enable -E to monitor tasks in this worker)
celery       | --- ***** ----- 
celery       |  -------------- [queues]
celery       |                 .> celery           exchange=celery(direct) key=celery

Here are more details on the console logs for docker-compose build/up and also failed background task https://gist.github.com/axilaris/095b4fd926df6a27b5e635fc1b51c854

2024-03-02 21:31:33,403 DEBUG 
def add(x, y):
    return 1
..
  File "/usr/local/lib/python3.9/contextlib.py", line 137, in __exit__
    self.gen.throw(typ, value, traceback)
  File "/usr/local/lib/python3.9/site-packages/kombu/connection.py", line 476, in _reraise_as_library_errors
    raise ConnectionError(str(exc)) from exc
kombu.exceptions.OperationalError: [Errno 111] Connection refused

I feel its something to do with my docker-compose.yaml or Dockerfile for backend and celery that is causing this problem.

I saw your files in github, you can do these:
first, in your init file in "backend/backend/init.py", set this: 

from .celery import app as celery_app

__all__ = ("celery_app",)

Set redis port in docker compose file and change your celery command:

redis: 
image: redis:alpine
ports:
  - "6379:6379"

celery:
restart: always
build:
  context: ./backend
command: celery -A backend worker -l info
volumes:
  - .:/django
container_name: celery  
depends_on:
  - redis

At the end you can set network for your docker compose, or check your docker log and the network.

You can also use this refrence: https://saasitive.com/tutorial/django-celery-redis-postgres-docker-compose/

Back to Top