Question

I’m trying to do something like this:

class A(models.Model):
    members = models.ManyToManyField(B)

class B(models.Model):
    pass

# not sure what the right query is here
results = A.objects.all().[members.extra(select={"extra_field": ...})?]

# I want to be able to write this using the result of my query:
for r in result:
    for m in r.members:
        print m.extra_field

Is it possible to populate this extra_field without creating a new query for every model B in members?

Answer 1

Until prefetching arrives in Django 1.4, this will get a little hairy. Hold onto your hat.

First, you need to be able to query over the table that links your two models. If you don’t want to explicitly define a through table, you can use an unmanaged model like so:

class AB(models.Model):
    a = models.ForeignKey('A')
    b = models.ForeignKey('B')

    class Meta:
        db_table = 'myapp_a_b'
        managed = False

Once you have this you can take a deep breath, hold your nose and do something like the following:

# Grab the links between A and B
a_b_relationships = AB.objects.all().values('a_id', 'b_id')

# Make an indexed reference for your B objects
all_b = B.objects.all().extra(select={'extra_field': ...})
b_dict = dict((b.pk, b) for b in all_b)

# Make a dict so that for any given A object we can immediately grab
# a list of its B objects
b_by_a = {}
for rel in a_b_relationships:
    a_id = rel['a_id']
    b = b_dict[rel['b_id']]

    if a_id not in b_by_a:
        b_by_a[a_id] = []

    if b not in b_by_a[a_id]
        b_by_a[a_id].append(b)

results = A.objects.all()

for r in result:
    members = b_by_a.get(r.id, [])

    for m in members:
        print m.extra_field

It’s nasty, but it works. Keep in mind that if the A and B tables start to get big then you’re going to run into performance issues – Django objects take up a lot of memory and can be very slow to iterate over. If you end up filtering or chunking A you’ll have to add the appropriate filters to the AB and B queries as well.

If anyone has a cleaner/more efficient way of doing this, I’d love to know!