How to efficiently combine Redis-based recommendation scoring with Django QuerySet for paginated feeds?

I'm building a marketplace app (think classified ads with negotiations) and trying to implement a personalized recommendation feed. I have a hybrid architecture question about the best way to handle this:

Current Setup:

  • Django backend with PostgreSQL for product data
  • Redis for user preferences, actions, and computed recommendation scores
  • Celery for background recommendation generation

The Challenge: I need to serve a paginated feed where the order is determined by Redis-based scoring (user preferences, trending items, negotiation activity), but the actual product data comes from Django models.

My Current Approach:

  1. Celery task generates ordered list of product IDs based on Redis metrics
  2. Cache this ordered list in Redis (e.g., [123, 456, 789, ...])
  3. For each page request, slice the cached ID list
  4. Use Django's Case/When to maintain the Redis-determined order:
preserved_order = models.Case(
    *[models.When(pk=pk, then=pos) for pos, pk in enumerate(page_product_ids)],
    output_field=models.IntegerField()
)

products = Product.objects.select_related('seller').filter(
    id__in=page_product_ids
).annotate(preserved_order=preserved_order).order_by('preserved_order')

Questions:

  1. Is using Case/When with enumerate() the most efficient way to preserve Redis-determined order in Django?
  2. Should I be caching the actual product data in Redis instead of just IDs?
  3. Any better patterns for this Redis scoring + Django data combination?
  4. How do you handle the "cold start" problem when recommendations aren't ready yet?

The feed needs to handle ~10k products with real-time scoring updates. Any architecture advice or alternative approaches would be greatly appreciated!

Tech Stack: Django 4.2, Redis, Celery, PostgreSQL

The Challenge: I need to serve a paginated feed where the order is determined by Redis-based scoring

In that case we thus know what items will be on the page. So what you can do is first paginate the list of ids to fetch instead, like:

from django.core.paginator import Paginator

page_product_ids = [123, 456, 789]

paginator = Paginator(page_product_ids, 2)
page = paginator.page(1)  # or any other page

preserved_order = models.Case(
    *[models.When(pk=pk, then=pos) for pos, pk in enumerate(page.object_list)],
    output_field=models.IntegerField()
)

products = (
    Product.objects.select_related('seller')
    .filter(id__in=page.object_list)
    .order_by(preserved_order.asc())
)

and now products is already paginated, given that all these ids thus still exist in the database. If you are not sure.

Вернуться на верх