Catch-all field for unserialisable data of serializer

I have a route where meta-data can be POSTed. If known fields are POSTed, I would like to store them in a structured manner in my DB, only storing unknown fields or fields that fail validation in a JSONField.

Let's assume my model to be:

# models.py
from django.db import models


class MetaData(models.Model):
  shipping_address_zip_code = models.CharField(max_length=5, blank=True, null=True)
  ...
  unparseable_info = models.JSONField(blank=True, null=True)

I would like to use the built-in serialisation logic to validate whether a zip_code is valid (5 letters or less). If it is, I would proceed normally and store it in the shipping_address_zip_code field. If it fails validation however, I would like to store it as a key-value-pair in the unparseable_info field and still return a success message to the client calling the route.

I have many more fields and am looking for a generic solution, but only including one field here probably helps in illustrating my problem.

 def validate_shipping_address_zip_code(self, value):
      if value >= 5:
        return value
      else:
          raise serializers.ValidationError("Message Here")

there's much more validators in serializer look into more detail https://www.django-rest-framework.org/api-guide/serializers/

from rest_framework import serializers
class MetaDataSerializer(serializers.ModelSerializer):
    unparseable_info = serializers.JSONField(required=False)
    class Meta:
        model = MetaData
        fields = ('shipping_address_zip_code', 'unparseable_info')
    
    def validate_shipping_address_zip_code(self, value):
        if len(value) > 5:
            raise serializers.ValidationError("Zip code is too long")
        return value
    
    def create(self, validated_data):
        unparseable_info = {}
        for key, value in self.initial_data.items():
            try:
                validated_data[key] = self.fields[key].run_validation(value)
            except serializers.ValidationError as exc:
                unparseable_info[key] = value
                validated_data.pop(key, None)
    
        instance = MetaData.objects.create(**validated_data)
    
        if unparseable_info:
            instance.unparseable_info = unparseable_info
            instance.save()
    
        return instance

You can use Django serializer that store fields that fail validation in JSONField.

Here is an example that worked for me:

from rest_framework import serializers

class MetaDataSerializer(serializers.ModelSerializer):
class Meta:
model = MetaData
fields = 'all'
def validate_shipping_address_zip_code(self, value):
    if len(value) > 5:
        raise serializers.ValidationError("Zip code must be 5 characters or less.")
    return value

def create(self, validated_data):
    unparseable_info = {}
    for field, value in self.initial_data.items():
        try:
            validated_data[field] = self.fields[field].run_validation(value)
        except serializers.ValidationError as e:
            unparseable_info[field] = value
    instance = MetaData.objects.create(**validated_data)
    if unparseable_info:
        instance.unparseable_info = unparseable_info
        instance.save()
    return instance

As you are looking for a generic solution, there are a few points that you should consider:

  1. Make sure not to place any model-level validations in your model as you want it to get saved irrespective of the validation status.
  2. Only validate on the serializer-level with custom validation methods.
  3. Make unparseable_info field ready-only as it is something we don't want the user to send but receive.
  4. Make use of the errors dictionary provided by the serializer as it gets populated with field-specific errors when we call is_valid.

This is how it might translate into code, inside models.py:

class MetaData(models.Model):
  shipping_address_zip_code = models.CharField(blank=True, null=True)
  ...
  unparseable_info = models.JSONField(blank=True, null=True)

then inside serializers.py:

class MetaDataSerializer(serializers.ModelSerializer):
    class Meta:
        model = MetaData
        read_only_fields = ('unparseable_info', )
        fields = '__all__'
    
    # Write validators for all of your fields.

finally inside your views.py method, something like this (you can do this inside serializer's save method as well):

meta_data = MetaDataSerializer(data=request.data)

if not meta_data.is_valid():
    meta_data.unparseable_info = meta_data.errors

meta_data.save()
# Return meta_data.data in JSONResponse.

You can create a custom serializer for this and use Django Rest Framework to validate and store the POSTed data.

First, create a serializer class for your model:

# serializers.py
from rest_framework import serializers
from .models import MetaData


class MetaDataSerializer(serializers.ModelSerializer):
  class Meta:
    model = MetaData
    fields = '__all__'

Then, in your view, you can validate the POSTed data using the serializer:

# views.py
from rest_framework import views, status
from rest_framework.response import Response
from .serializers import MetaDataSerializer
from .models import MetaData


class MetaDataView(views.APIView):
  def post(self, request, format=None):
    serializer = MetaDataSerializer(data=request.data)
    if serializer.is_valid():
      shipping_address_zip_code = serializer.validated_data.get('shipping_address_zip_code', None)
      if shipping_address_zip_code and len(shipping_address_zip_code) <= 5:
        # store the data in the shipping_address_zip_code field
        serializer.save()
        return Response(serializer.data, status=status.HTTP_201_CREATED)
      else:
        # store the unparseable data in the unparseable_info field
        unparseable_data = {'shipping_address_zip_code': shipping_address_zip_code}
        meta_data = MetaData.objects.create(unparseable_info=unparseable_data)
        return Response({'id': meta_data.id}, status=status.HTTP_201_CREATED)
    else:
      # store the unparseable data in the unparseable_info field
      unparseable_data = request.data
      meta_data = MetaData.objects.create(unparseable_info=unparseable_data)
      return Response({'id': meta_data.id}, status=status.HTTP_201_CREATED)

This way, if the shipping_address_zip_code field is valid, it will be stored in the shipping_address_zip_code field, otherwise it will be stored in the unparseable_info field along with any other unparseable data.

Back to Top