Django Celery Beat SQS slow scheduling

Beat seems to be sending the messages into SQS very slowly, about 100/minute. Every Sunday I have a sendout to about 16k users, and they're all booked for 6.30pm. Beat starts picking it up at the expected time, and I would expect a huge spike in messages coming into SQS at that time, but it takes its time, and I can see on the logs that the "Sending tasks x..." goes on for a few hours. I expect ~16k messages to go out around 6.30pm, and for the number of messages processed and deleted to pick up as the autoscale sets in.

I have autoscaling on for my Celery workers, but because the number of messages doesn't really ever spike, the workers don't really scale until later, when the messages start backing up a bit.

I'm really puzzled by this behaviour, anyone there know what I could be missing? I'm running celery with, some cron tab tasks but this one task in specific is a PeriodicTask

celery_beat: celery -A appname beat --loglevel=INFO

enter image description here

enter image description here

Beat is single-threaded, so when django_celery_beat hands it thousands of due rows it must loop, update each row, and push to SQS one HTTP call at a time. That explains the 100/minute ceiling you’re seeing.

This is a sample code that you can see how to use it:

@app.task
def schedule_sunday_run():
    users = list(fetch_recipients())
    group(send_email.s(u.id) for u in users).apply_async()
Вернуться на верх