Setup logging of Celery in Django correctly - how?

Setting up celery on our production server seems to be more complicated than it should be - and the problem is logging. Our setup is like this:

  • Server is Ubuntu 20.04.5 LTS
  • We run Django 3.2.8 in it's own user and group named "django"
  • The Celery process also has it's own user and group named "celery".
  • Celery is set up as a service and logs to /var/log/celery
  • In celery.py we have set djangoConf pointing to the main settings.py

On startup Celery tries to touch a django-logfile and - missing the permissions to do so - returns an error:

Dec 16 10:57:27 h2989788 celery[128418]: Usage: celery [OPTIONS] COMMAND [ARGS]...
Dec 16 10:57:27 h2989788 celery[128418]: Try 'celery --help' for help.
Dec 16 10:57:27 h2989788 celery[128418]: Error: Invalid value for '-A' / '--app':
Dec 16 10:57:27 h2989788 celery[128418]: Unable to load celery application.
Dec 16 10:57:27 h2989788 celery[128418]: While trying to load the module theApp the following error occurred:
Dec 16 10:57:27 h2989788 celery[128418]: Traceback (most recent call last):
Dec 16 10:57:27 h2989788 celery[128418]:   File "/usr/lib/python3.8/logging/config.py", line 563, in configure
Dec 16 10:57:27 h2989788 celery[128418]:     handler = self.configure_handler(handlers[name])
Dec 16 10:57:27 h2989788 celery[128418]:   File "/usr/lib/python3.8/logging/config.py", line 744, in configure_handler
Dec 16 10:57:27 h2989788 celery[128418]:     result = factory(**kwargs)
Dec 16 10:57:27 h2989788 celery[128418]:   File "/usr/lib/python3.8/logging/handlers.py", line 148, in __init__
Dec 16 10:57:27 h2989788 celery[128418]:     BaseRotatingHandler.__init__(self, filename, mode, encoding, delay)
Dec 16 10:57:27 h2989788 celery[128418]:   File "/usr/lib/python3.8/logging/handlers.py", line 55, in __init__
Dec 16 10:57:27 h2989788 celery[128418]:     logging.FileHandler.__init__(self, filename, mode, encoding, delay)
Dec 16 10:57:27 h2989788 celery[128418]:   File "/usr/lib/python3.8/logging/__init__.py", line 1147, in __init__
Dec 16 10:57:27 h2989788 celery[128418]:     StreamHandler.__init__(self, self._open())
Dec 16 10:57:27 h2989788 celery[128418]:   File "/usr/lib/python3.8/logging/__init__.py", line 1176, in _open
Dec 16 10:57:27 h2989788 celery[128418]:     return open(self.baseFilename, self.mode, encoding=self.encoding)
Dec 16 10:57:27 h2989788 celery[128418]: PermissionError: [Errno 13] Permission denied: '/home/django/FLY/logs/myrate.log'
Dec 16 10:57:27 h2989788 celery[128418]: The above exception was the direct cause of the following exception:
Dec 16 10:57:27 h2989788 celery[128418]: Traceback (most recent call last):
Dec 16 10:57:27 h2989788 celery[128418]:   File "/home/django/env/lib/python3.8/site-packages/kombu/utils/imports.py", line 56, in symbol_by_name
Dec 16 10:57:27 h2989788 celery[128418]:     module = imp(module_name, package=package, **kwargs)
Dec 16 10:57:27 h2989788 celery[128418]:   File "/home/django/env/lib/python3.8/site-packages/celery/utils/imports.py", line 105, in import_from_cwd
Dec 16 10:57:27 h2989788 celery[128418]:     return imp(module, package=package)
Dec 16 10:57:27 h2989788 celery[128418]:   File "/usr/lib/python3.8/importlib/__init__.py", line 127, in import_module
Dec 16 10:57:27 h2989788 celery[128418]:     return _bootstrap._gcd_import(name[level:], package, level)
Dec 16 10:57:27 h2989788 celery[128418]:   File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
Dec 16 10:57:27 h2989788 celery[128418]:   File "<frozen importlib._bootstrap>", line 991, in _find_and_load
Dec 16 10:57:27 h2989788 celery[128418]:   File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
Dec 16 10:57:27 h2989788 celery[128418]:   File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
Dec 16 10:57:27 h2989788 celery[128418]:   File "<frozen importlib._bootstrap_external>", line 848, in exec_module
Dec 16 10:57:27 h2989788 celery[128418]:   File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
Dec 16 10:57:27 h2989788 systemd[1]: celery.service: Control process exited, code=exited, status=2/INVALIDARGUMENT
Dec 16 10:57:27 h2989788 celery[128418]:   File "/home/django/FLY/theApp/__init__.py", line 1, in <module>
Dec 16 10:57:27 h2989788 celery[128418]:     from .celery import app as celery_app
Dec 16 10:57:27 h2989788 celery[128418]:   File "/home/django/FLY/theApp/celery.py", line 18, in <module>
Dec 16 10:57:27 h2989788 celery[128418]:     configurations.setup()
Dec 16 10:57:27 h2989788 celery[128418]:   File "/home/django/env/lib/python3.8/site-packages/configurations/__init__.py", line 31, in setup

Workaround: change the permissions of the log file, which belongs to user django, so that user celery can write to it. But as we're using a rotating logger, this only helps until the next log-file is created.

Solutions we thought of:

  • Changing the access-rights of the file every time a new log-file is created: probably the worst way of handling this
  • Using the same user for the django- and celery-processes: smells like the last resort
  • Creating a specific run-configuration in settings.py overwriting the logging handlers like so:
class CeleryProd(Prod):
    def __init__(self) -> None:
        super().__init__()
        self.LOGGING['handlers']={}

But somehow every of these solutions seems like a workaround for me. There must be a way of doing it right I think? What's the correct way of doing this - if this even exists?

Back to Top