Why do custom managers break related object queries?

I know this is a long one, but I promise its worth a read: useful solutions, and learning about the inner workings of Django!

The setup

I came across this issue while working with a logged model with a ForeignKey relationship to another logged model. When deserializing an instance, I want to include its related instances.

Note that I use the django_rest_framework serializers and specify the related fields in the Meta.fields option. This isn't very relevant so I won't include the code but can if requested.

Tiny Example:

models.py

class A(models.Model):
    ...
class B(models.Model):
    a = models.models.ForeignKey(A, on_delete=models.CASCADE)
    ...

Deserializing an instance of A should return something such as: {..., 'b_set': [3, 6, 7], ...}. We get a possibly empty array of related IDs.

The issue

The issue arises when adding custom managers:

First, we define a LoggedModelManager, all it does is filter out the logs from the current instances.
Second, we define the models A and its logs ALog. A gets the custom manager -> to ensure A.objects.all() only returns instances of A, not ALog, and then we ensure ALog has the default manager (django takes care of filtering away the non-log instances).
Third we do the same for B. Notice that B has a ForeignKey to A.

# Defining the Manager
class LoggedModelManager(models.Manager):

     # Save the name of the logging model, which are `ALog` and `BLog` in our example.
    def __init__(self, *args, **kwargs):
        logging_model = kwargs.pop('logging_model')
        self.logging_model_name = logging_model.lower()
        super().__init__(*args, **kwargs)

    # Here we filter out the logs from the logged instances.
    def get_queryset(self) -> models.QuerySet:
        return super().get_queryset().filter(**{self.logging_model_name+"__isnull": True})

# Define A and its log
class A(models.Model):
    objects = LoggedModelManager(logging_model='ALog')
    ...
class ALog(A):
    objects = models.Manager()
    instance = models.ForeignKey(A, on_delete=models.CASCADE)

# Define B and its log
class B(models.Model):
    objects = LoggedModelManager(logging_model='BLog')
    ...
class BLog(B):
    objects = models.Manager()
    instance = models.ForeignKey(B, on_delete=models.CASCADE)

The outcome: A will deserialize to: {..., 'alog_set': [...], ...}. But notice that b_set is missing!

My (uninformed) solution.

After many hours of frustration and reading the docs, the following rewrite of B fixed this issue:

Note that BLog has to overwrite these modifications too but I've omitted this.

class B(models.Model):
    objects = LoggedModelManager(logging_model='BLog')
    objects_as_related = models.Manager()
    ...
    class Meta:
        base_manager_name = 'objects'
        default_manager_name = 'objects_as_related'

This works: A now deserializes to {..., 'alog_set': [...], 'b_set': [...] ...}. Note that the default_manager_name manager cannot be my custom manager.

My Question

Why does adding a custom manager break this? It returns a QuerySet, so why not return an empty 'b_set': [] instead of it not existing?
The documentation on Default Managers is super confusing, it says that:

"By default, Django uses an instance of the Model._base_manager manager class when accessing related objects, not the _default_manager on the related object."

But my example appears to show the contrary? Am I miss-reading the documentation?