Django-celery-beat loading task but celery worker not receiving it

I've been having these issues on my celery beat and worker where my celery beat creates task, but celery worker does not receive it . I am using elasticmq as the broker My celery beat logs docker exec -it app-1 celery -A app beat --loglevel info celery beat v5.3.6 (emerald-rush) is starting. __ - ... __ - _ LocalTime -> 2024-03-09 16:25:41 Configuration -> . broker -> sqs://user:**@sqs:9324// . loader -> celery.loaders.app.AppLoader . scheduler -> celery.beat.PersistentScheduler . db -> celerybeat-schedule . logfile -> [stderr]@%INFO . maxinterval -> 5.00 minutes (300s) [2024-03-09 16:25:41,744: INFO/MainProcess] beat: Starting... [2024-03-09 16:25:41,821: INFO/MainProcess] Scheduler: Sending due task beat_task (app.tasks.beat_task) [2024-03-09 16:25:51,782: INFO/MainProcess] Scheduler: Sending due task beat_task (app.tasks.beat_task)

My celery worker log

docker exec -it kole-app-api-app-1 celery -A app worker --loglevel info/usr/local/lib/python3.11/site-packages/celery/platforms.py:829: SecurityWarning: You're running the worker with superuser privileges: this isabsolutely not recommended!

Please specify a different user using the --uid option.

User information: uid=0 euid=0 gid=0 egid=0

warnings.warn(SecurityWarning(ROOT_DISCOURAGED.format(

-------------- celery@b4d9d70aafdc v5.3.6 (emerald-rush)--- ***** ------- ******* ---- Linux-5.10.16.3-microsoft-standard-WSL2-x86_64-with 2024-03-09 16:26:10

  • *** --- * ---

  • ** ---------- [config]

  • ** ---------- .> app:         app:0x7fd2fe122f90

  • ** ---------- .> transport:   sqs://user:**@sqs:9324//

  • ** ---------- .> results:     disabled://

  • *** --- * --- .> concurrency: 6 (prefork)-- ******* ---- .> task events: OFF (enable -E to monitor tasks in this worker)--- ***** ------------------- [queues].> celery           exchange=celery(direct) key=celery

[tasks]. app.tasks.beat_task. app.tasks.web_task

[2024-03-09 16:26:11,394: WARNING/MainProcess] /usr/local/lib/python3.11/site-packages/celery/worker/consumer/consumer.py:507: CPendingDeprecationWarning: The broker_connection_retry configuration setting will no longer determine whether broker connection retries are made during startup in Celery 6.0 and above. If you wish to retain the existing behavior for retrying connections on startup, you should set broker_connection_retry_on_startup to True. warnings.warn(

[2024-03-09 16:26:11,450: INFO/MainProcess] Connected to sqs://user:**@sqs:9324// [2024-03-09 16:26:11,451: WARNING/MainProcess] /usr/local/lib/python3.11/site-packages/celery/worker/consumer/consumer.py:507: CPendingDeprecationWarning: The broker_connection_retry configuration setting will no longer determine whether broker connection retries are made during startup in Celery 6.0 and above. If you wish to retain the existing behavior for retrying connections on startup, you should set broker_connection_retry_on_startup to True. warnings.warn(

[2024-03-09 16:26:11,509: INFO/MainProcess] celery@b4d9d70aafdc ready.

It does not pick up the tasks from celery beat, here is the snippet of the task


app = Celery("app")
logger = get_task_logger(__name__)

CELERY_CONFIG = {
    "CELERY_TASK_SERIALIZER": "json",
    "CELERY_ACCEPT_CONTENT": ["json"],
    "CELERY_RESULT_SERIALIZER": "json",
    "CELERY_RESULT_BACKEND": None,
    "CELERY_ENABLE_UTC": True,
    "CELERY_ENABLE_REMOTE_CONTROL": False,
}

BROKER_URL = "sqs://user:password@sqs:9324/"

CELERY_CONFIG.update(
    {
        "BROKER_URL": BROKER_URL,
        "BROKER_TRANSPORT": "sqs",
        "BROKER_TRANSPORT_OPTIONS": {
            "region": "us-west-2",
            "use_ssl": False,
            "visibility_timeout": 3600,
            "polling_interval": 60,
        },
    }
)

app.conf.update(**CELERY_CONFIG)
app.config_from_object("django.conf:settings", namespace="CELERY")

and here is the task,

@celery.app.task()
def beat_task() -> None:
    logger.info("Starting beat task...")
    time.sleep(2)
    logger.info("Done beat task.")

I am not getting the logger info print result on worker

Tried connecting celery worker and beat, did not work

the celery.app object is not defined here.the Celery object is here initialized with the name app and not celery.app. so change the decorator to @app.task().

app = Celery("app")
logger = get_task_logger(__name__).....

@app.task()
def beat_task().....

update any other references from celery.app in your code to app

Back to Top