Django.db.utils.IntegrityError: UNIQUE constraint failed: items_auction.auction_id only when bulk_create

I have a model called "Auction", it has a unique pk field called auction_id. Now i have a lot of data (over 50k records) and the problem is that when the database is not empty and im trying to upload another batch of records using bulk_create the error pops up.

Here is my script for filling data

auctions = [
    Auction(
        auction_id=row[0],
        price=row[1],
        quantity=row[2],
        item_id=row[3],
        auctioned_item=Item.objects.get(item_id=row[3]) if Item.objects.filter(item_id=row[3]).exists() else None,
        realm_id = realm_id
    )
    for row in reader
]
items = []
for auction in auctions:
    if Auction.objects.filter(pk=auction.auction_id).exists():
        Auction.objects.get(pk=auction.auction_id).delete()
    try:
        item = auction.auctioned_item
        if item not in items:
            item.last_auctioned_date = datetime.now()
            items.append(item)
    except:
        pass

Auction.objects.bulk_create(objs=auctions, ignore_conflicts=True)
Item.objects.bulk_update(items, ["last_auctioned_date"])

Here is the full error

Traceback (most recent call last):
  File "C:\Users\kacpe\Desktop\praca_inz\venv\lib\site-packages\django\db\backends\utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
  File "C:\Users\kacpe\Desktop\praca_inz\venv\lib\site-packages\django\db\backends\sqlite3\base.py", line 423, in execute
    return Database.Cursor.execute(self, query, params)
sqlite3.IntegrityError: UNIQUE constraint failed: items_auction.auction_id

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "C:\Users\kacpe\Desktop\praca_inz\venv\lib\site-packages\django\db\models\manager.py", line 85, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "C:\Users\kacpe\Desktop\praca_inz\venv\lib\site-packages\django\db\models\query.py", line 502, in bulk_create
    returned_columns = self._batched_insert(
  File "C:\Users\kacpe\Desktop\praca_inz\venv\lib\site-packages\django\db\models\query.py", line 1293, in _batched_insert
    self._insert(item, fields=fields, using=self.db, ignore_conflicts=ignore_conflicts)
  File "C:\Users\kacpe\Desktop\praca_inz\venv\lib\site-packages\django\db\models\query.py", line 1270, in _insert
    return query.get_compiler(using=using).execute_sql(returning_fields)
  File "C:\Users\kacpe\Desktop\praca_inz\venv\lib\site-packages\django\db\models\sql\compiler.py", line 1416, in execute_sql
    cursor.execute(sql, params)
  File "C:\Users\kacpe\Desktop\praca_inz\venv\lib\site-packages\django\db\backends\utils.py", line 98, in execute
    return super().execute(sql, params)
  File "C:\Users\kacpe\Desktop\praca_inz\venv\lib\site-packages\django\db\backends\utils.py", line 66, in execute
    return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
  File "C:\Users\kacpe\Desktop\praca_inz\venv\lib\site-packages\django\db\backends\utils.py", line 75, in _execute_with_wrappers
    return executor(sql, params, many, context)
  File "C:\Users\kacpe\Desktop\praca_inz\venv\lib\site-packages\django\db\backends\utils.py", line 79, in _execute
    with self.db.wrap_database_errors:
  File "C:\Users\kacpe\Desktop\praca_inz\venv\lib\site-packages\django\db\utils.py", line 90, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "C:\Users\kacpe\Desktop\praca_inz\venv\lib\site-packages\django\db\backends\utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
  File "C:\Users\kacpe\Desktop\praca_inz\venv\lib\site-packages\django\db\backends\sqlite3\base.py", line 423, in execute
    return Database.Cursor.execute(self, query, params)
django.db.utils.IntegrityError: UNIQUE constraint failed: items_auction.auction_id

This only happens when using bulk_create, when using get_or_create or just create the error doesnt show up. The weirder thing is that when i use bulk_create(ignore_conflicts=True) and check the records for duplicates afterwards, there are none and everything seems fine. So what's the problem here?

Back to Top