Django 5.0.x async with raw query set & iterator
I have a raw query set that returns a few millions of elements. Actually, I use the iterator()
function of django to improve performances.
The problem I have here, is that I want to make the function where the query is called asynchronous. Because of that, I can't make Django raw queryset anymore because I get this error : django.core.exceptions.SynchronousOnlyOperation: You cannot call this from an async context - use a thread or sync_to_async.
Using sync_to_async makes the use of iterator
impossible. There is the aiterator()
function, but it doesn't work on raw query set.. How can I use iterator with a raw query set in an asynchronous context ?
Code :
def _get_elements_to_process(self):
return Elements.objects.raw(
"""
My query
""",
)
async def fetch_data(self):
for element in _get_elements_to_process().iterator():
# make some asynchronous action