Django Materialized View Refreshing Celery Task is freezing in DB layer for larger dataset
In my application, written in Django 5.2, I want to aggregate static data and store it as a materialized view in PostgreSQL 16.10. I can create it as my own migration or using the django-materialized-view library. I have no problems creating it.
However, when I call the celery task, which should refresh the view after updating the data, it “freezes” when I enable updates for all three carriers. On the other hand, if I remove the third carrier (whose data weight is approximately 95% of the total of the three), the refresh task runs without any problems. I could blame this on the giant size of the data, but if I run the update only for this giant carrier or write the refresh command myself in DBMS, it executes successfully in 20-30 seconds.
The Celery worker that performs update and refresh tasks has concurrency=1 (the refresh task is, of course, the last in the queue), and the configuration of work_mem, maintanence_work_mem, and shared_buffers in the database should definitely be able to handle this task. During the update, no other queries are executed in the database, and the refresh is CONCURRENTLY.
You can find my project in the GitHub repository: text
If you contact me, I can also provide a sample version of the .env file so that you can run the project locally.