Django ORM not releasing memory even when garbage collector is called explicitly

It seems like ORM objects are not releasing memory - please refer code below.

I tried various approaches like, but nothing helped:

  1. Manually call gc.collect()
  2. Manually disable and enable gc
  3. Use queryset
  4. Use iterators
  5. Use lists

In actual case, I query about 60K articles, and would like the memory to be released as soon as I am out of the function. The memory is not released even after days. So, I guess its not an issue with garbage collector.

Please suggest.

import gc
import os
import django
import psutil

os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'mysite.settings')
django.setup()

from api.models import Article

def fetch_articles():
    # gc.disable()  # no help
    articles = Article.objects.order_by('-id')[:10].iterator()
    # articles = Article.objects.order_by('-id')[:10]  # similar memory consumption
    # articles = list(Article.objects.order_by('-id')[:10])  # similar memory consumption

    for article in articles:
        pass
    del article
    del articles
    # gc.enable()  # no help
    gc.collect()


# gc.collect()  # no help
process = psutil.Process(os.getpid())
print(process.memory_info().rss / (1024 * 1024), "MB")  # 41.203125 MB
fetch_articles()
# gc.collect()  # no help
print(process.memory_info().rss / (1024 * 1024), "MB")  # 44.21875 MB
Back to Top