Django import-export import foreign keys that do not exist
Being in a rush I'm having a hard time understanding the import concept.
Where is the final board creation missing? i.e. Board(person=p, organization=p)
Model
class Person(BaseModel):
organizations = models.ManyToManyField("Organization",
blank=True,
through="Board")
class Organization(BaseModel):
people = models.ManyToManyField("Person", blank=True, through="Board")
class Board(BaseModel):
person = models.ForeignKey(Person, on_delete=models.CASCADE)
organization = models.ForeignKey(Organization, on_delete=models.CASCADE)
Test
from django.test import TestCase
import tablib
from import_export import resources
class BoardResource(resources.ModelResource):
def before_import_row(self, row, **kwargs):
org_name_1 = row["organization"]
o=Organization.objects.get_or_create(name_1=org_name_1, defaults={"name_1": org_name_1})
person_firstname = row["person"]
p=Person.objects.get_or_create(firstname=person_firstname, defaults={"firstname": person_firstname})
class Meta:
model = Board
dataset = tablib.Dataset(['','john','acme'], headers=['id','person','organization'])
class TestCase(TestCase):
def test_basic_import(self):
board_resource = BoardResource()
result = board_resource.import_data(dataset, dry_run=False)
print(result.totals)
assert not result.has_errors()
The documentation points to this thread though I'm unable to apply anything to my case
I'm wondering if your data model can be improved. My assumptions may be wrong, but I think they are as follows:
- A Person can be in N Organizations.
- A Board instance provides the 'through' relationship and has a reference to a single Person, and single Organization.
Therefore you don't need to declare a reference to Person from Organization.
class Person(BaseModel):
organizations = models.ManyToManyField("Organization",
blank=True,
through="Board")
class Organization(BaseModel):
name = models.CharField(max_length=255)
class Board(BaseModel):
person = models.ForeignKey(Person, on_delete=models.CASCADE)
organization = models.ForeignKey(Organization, on_delete=models.CASCADE)
Now you import a Board instance which contains a reference to a Person (by first name) and an Organization by name. The Person and / or Organization may not exist and may need to be created during import.
The Board instance may or may not exist during import (it will be identified by the 'id' field), and either a new or existing Board instance will be loaded here.
However, now you need to implement the logic to create the Person / Organization and link it with the instance.
UPDATE I rewrote my original answer as follows:
The best way to create an instance is to create a subclass of ForeignKeyWidget
which handles the creation of the instance if it does not exist:
class PersonForeignKeyWidget(ForeignKeyWidget):
def clean(self, value, row=None, **kwargs):
try:
val = super().clean(value)
except Person.DoesNotExist:
val = Person.objects.create(firstname=row['person'])
return val
Do the same for Organization
as well.
Then declare these widgets in your Resource
:
person = fields.Field(
widget=PersonForeignKeyWidget(Person, "firstname"))
organization = fields.Field(
widget=OrganizationForeignKeyWidget(Organization, "name"))
Now the import logic will continue and persist the Board instance with the correct relations.
There's a few questions like what if people share a firstname... and this obviously doesn't handle creation of other fields (e.g. like what's in BaseModel).
If it is still failing with 'invalid', you will need to inspect the source of the error. See docs for techniques on reading errors.
As always with import_export the best way to understand it is to set a debugger and step through an import.
There are two soluions:
I.Changin your code and using django model
it seems that the issue you're facing is related to the creation of the Board instances after importing data for Person and Organization. In the before_import_row method, you are correctly retrieving or creating the Person and Organization instances, but you are missing the actual creation of the Board instance that links the person and organization.
Here's how you can adjust the code to ensure that a Board instance is created after the Person and Organization objects are successfully imported or retrieved:
Solution You need to create a Board instance inside the before_import_row method. Here's an updated version of your BoardResource class:
from import_export import resources
from django.test import TestCase
import tablib
from .models import Person, Organization, Board
class BoardResource(resources.ModelResource):
def before_import_row(self, row, **kwargs):
org_name_1 = row["organization"]
# Retrieve or create the organization
o, created_org = Organization.objects.get_or_create(name_1=org_name_1, defaults={"name_1": org_name_1})
person_firstname = row["person"]
# Retrieve or create the person
p, created_person = Person.objects.get_or_create(firstname=person_firstname, defaults={"firstname": person_firstname})
# Create the Board instance that connects the Person and Organization
Board.objects.get_or_create(person=p, organization=o)
class Meta:
model = Board
# Sample dataset for testing
dataset = tablib.Dataset(['','john','acme'], headers=['id','person','organization'])
class TestCase(TestCase):
def test_basic_import(self):
board_resource = BoardResource()
result = board_resource.import_data(dataset, dry_run=False)
print(result.totals)
assert not result.has_errors()
I.Django model is using sqlite so you could use it directly. (This is a sample)
import sqlite3
connection = sqlite3.connect(<DIR_TO_DATABASE>,timeout=<secconds>)
connection.execute(f'''CREATE TABLE IF NOT EXISTS ....''')
connection.commit()
connection.close()
As you see you could do it by running pure sql.