Automatically Retrying Failed Celery Tasks
In this article, we'll look at how to automatically retry failed Celery tasks.
After reading, you should be able to:
- Retry a failed Celery task with both the
retrymethod and a decorator argument
- Use exponential backoff when retrying a failed task
- Use a class-based task to reuse retry arguments
You can find the source code for this article on GitHub.
Let's assume we have a Celery task like this:
@shared_task def task_process_notification(): if not random.choice([0, 1]): # mimic random error raise Exception() requests.post('https://httpbin.org/delay/5')
In the real world this may call an internal or external third-party service. Regardless of the service, assume it's very unreliable, especially at peak periods. How can we handle failures?
Back to Top
It's worth noting that many Celery beginners get confused as to why some articles use
app.taskwhile others use
shared_tasklets you define Celery tasks without having to import the Celery instance, so it can make your task code more reusable.