Django task to created objects in database systematically failed on the first record

I use Django (2.2.5), Docker, celery and Postgresql database. I have write a celery task that query database first, create a dataframe with results of the query and update_or_create objects in databases based on this dataframe.

code simplified

# load queries definition
    queries = dcf_queries_definitions() # this line query the database
    # print(queries)
    if not queries.empty:
        for index, row in queries.iterrows():
            # multiple call to api of another app to look for missing data
            # return **df** dataframe that will be used to create objects in database
        records = df.to_dict(orient='records')
        for record in records:
            try:
                DataCorrectionForm.objects.create(
                    dcf_ide=record['dcf_ide'],
                    category=record['category'],
                    crf=record['crf'], 
                    crf_ide=record['crf_ide'],
                    patient=record['patient'],
                    record_date=record['record_date'],
                    field_name=record['field_name'],
                    field_label=record['field_label'],
                    message=record['message'],
                    field_value=record['field_value'],
                    dcf_status=record['dcf_status'],
                    query_id=record['query_id'],
                    deactivated=record['deactivated'],
                    comments=record['comments']
                )  
            except Exception as e:
                print(record)
                print('Exception raised when trying to update object',e)
                continue 

But I have a OperationalError that systematically raised on the first objects to be created or updated. The message in not very explicit for me.

OperationalError('server closed the connection unexpectedly\n\tThis probably means the server terminated abnormally\n\tbefore or while processing the request.\n')

I controlled data I try to insert in the model and there is no problem. If I upload the same datas from a csv file to insert in my database, there is no problem.

I suspect problem with transaction between querying database and trying to insert in database in the same task.

If I try to create object with the create method (not update_or_create method) and captur the error, the error is connection already closed.

I initially open a post here but got no answer but analysis of the issue is now quite different.

I would appreciate some help and advice.

Back to Top