Should I reach into the Django _prefetched_objects_cache to solve an N+1 query?

I have the following Django template code with an N+1 query:

{% for theobject in objs %}
  {% for part in theobject.parts_ordered %}
      <li>{{ part }}</li>
  {% endfor %}
{% endfor %}

Here is parts_ordered on TheObject:

  class TheObject:
    # ...
    def parts_ordered(self) -> list["Part"]:
        return self.parts.all().order_by("pk")

And here is the Part object:

  class Part:
    # ...
    theobject = models.ForeignKey(
        TheObject, on_delete=models.CASCADE, related_name="parts"
    )

and here is the prefetch getting objs:

    ofs = ObjectFormSet(
        queryset=TheObject.objects
        .filter(objectset=os)
        .prefetch_related("parts")
    )

I think the order_by("pk") disrupts the prefetch.

This is what chatgpt recommends, and it works (no more N+1 queries, results seem the same):

  class TheObject:
    # ...
    def parts_ordered(self) -> list["Part"]:
        if (
            hasattr(self, "_prefetched_objects_cache")
            and "parts" in self._prefetched_objects_cache
        ):
            # Use prefetched data and sort in Python
            return sorted(
                self._prefetched_objects_cache["parts"], key=lambda cc: cc.pk
            )

        # Fallback to querying the DB if prefetching wasn’t used
        return self.parts.all().order_by("pk")

Should I rely on _prefetched_objects_cache? Is there a better way?

I wouldn't use private methods, or attributes in your case. As a rule, library authors mark them as non-public for a reason. Most often they are implementation details that may change in some next version, which may cause your code to stop working.

In your case, you can make things simpler, but still solve the N + 1 problem. For this you can use Prefetch object and this query:

from django.db import models  
  
  
ofs = ObjectFormSet(  
    queryset=TheObject.objects  
    .filter(objectset=os)  
    .prefetch_related(  
        models.Prefetch(  
            lookup='parts',  
            queryset=Part.objects.order_by('pk'),  
            to_attr='parts_ordered',  
        ),  
    )  
)

This will give similar results to yours and should improve overall performance a bit, since the database is doing the sorting and not python.

Вернуться на верх