Django filter an m2m relation by a list of inputs (which must all match)
Let's take some Store
and Book
models as examples:
class Book(Model):
title = CharField(...)
...
class Store(Model):
books = ManyToManyField('Book', blank=True, related_name='stores')
....
I receive a list of book titles and must return stores linked to those books.
I need the option for both an AND
query and an OR
query.
The OR
query is rather simple; we only need a store to match once:
Store.objects.filter(book__title__in=book_titles)
However the AND
query seems tricky. Perhaps I am simply too deep to notice, but so far I have only managed by chaining queries which is not very good, at least performance-wise.
from django.db.models import Q
filtering = Q()
for book_title in book_title_list:
filtering &= Q(id__in=Book.objects.get(title=book_title).stores)
Store.objects.filter(filtering)
This effectively creates an OUTER JOIN
and a SELECT
within the WHERE
clause for every book title, which at 2 or 3 is not much but definitely not advisable when user input is not limited.
Without explicitly looping and adding Q
objects like this I have yet to obtain a query that actually works.
More often than not, the query either only evaluates a single line of the m2m relation or behaves similarly to the OR
query. As a reminder, the AND
query needs all returned stores to be linked to all of the books whose titles were given.
I have tried meddling with the Subquery
, CASE
, WHERE
and annotate
classes and methods provided by Django but I am by no means an expert and have only experienced failure.
At some point I was thinking that using annotate
with an incrementing value for each match (given that the length of the list is known) could work, but have yet to find some way to actually make it work.
For added difficulty, I also cannot loop over / access any of the querysets since the number of items is unknown but is definitely 100k+.
You can check if all book
s match with:
from django.db.models import Count
book_titles_set = set(book_titles)
Store.objects.filter(book__title__in=book_titles_set).annotate(
nbook=Count('book')
).filter(nbook=len(book_titles_set))