Duplicate MongoDB Records Issue with update_one and upsert under High Traffic

I am currently facing a challenging issue in my Django application integrated with MongoDB, particularly when dealing with high traffic and concurrent users. Despite using update_one with upsert, I'm observing instances where double (at times tripple) records are being created intermittently under these conditions.

Below is a simplified example of the code snippet I'm using:

from pymongo import MongoClient

from rest_framework.views import APIView
from rest_framework import status
from rest_framework.response import Response

from django.conf import settings


# MongoDB connection setup
client = MongoClient(settings.MONGO_CONNECTION_URL)
db = client[settings.DATABASE_NAME]
collection = db[settings.COLLECTION_NAME]


class UserDataView(APIView):
    def post(self, request) -> Response:
        try:
            # Using update_one with upsert to handle user data updates
            collection.update_one(
                {'user_id': request.user.pk},
                {'$set': {'data': request.data["data"]}},
                upsert=True
            )
            
            # Returning a successful response with a data-related message
            return Response(
                {'message': 'User data successfully updated or created'},
                status=status.HTTP_200_OK
            )
        except KeyError as e:
            # Logging and handling potential errors
            logger.error(e)
            
            # Returning a response for a bad request with a data-related message
            return Response(
                {'message': 'Bad request. Please provide the required data'},
                status=status.HTTP_400_BAD_REQUEST
            )

Even with the upsert=True option, duplicate records are created sporadically, and this seems to happen during periods of high traffic and concurrent user activity.

I've ensured that the user_id is correctly set as the unique identifier, and there are no apparent errors in the MongoDB logs. This behavior is challenging to reproduce, making it difficult to identify the root cause.

Are there any considerations or best practices I should be aware of when using update_one with upsert in scenarios with high traffic and concurrent users? Any insights on how to address or troubleshoot this issue would be highly appreciated.

Back to Top