Question

Been writing some raw SQL queries after noticing how inefficient some of Django’s built-in queries were. I’m trying to loop through the QuerySet result and group them into categories (I’m aware of the regroup template tag, this doesn’t work for me – I need to be able to access the separate groups independently). Here’s my code:

m = Media.objects.raw('SELECT * FROM table') # query simplified for sake of example

media_items = {'aim-icons' : [], 'banners' : [], 'hi-res-photos' : [], 'photos' : [], 'print-ads' : [], 'videos' : [], 'wallpapers' : [] }

for item in m:
    media_items[item.type_slug].append(item)

This gives me what I want (eg a list that I can access like media_items['wallpapers']) but it runs a database query for every iteration to fetch the type_slug field. I tried adding m = list(m) before the loop, no effect.

Can anyone help me out here? This seems like it should be simple.

Thanks,
Matt

Answer 1

Edit:

The issue breaks down here to how Django’s raw() method works. It returns model instances (which had properties you were accessing, resulting in the extra query).

The proper tools here are connection.cursor(), cursor.execute() and cursor.fetchall(). Here’s the example from the docs:

def my_custom_sql():
    from django.db import connection, transaction
    cursor = connection.cursor()

    # Data modifying operation - commit required
    cursor.execute("UPDATE bar SET foo = 1 WHERE baz = %s", [self.baz])
    transaction.commit_unless_managed()

    # Data retrieval operation - no commit required
    cursor.execute("SELECT foo FROM bar WHERE baz = %s", [self.baz])
    row = cursor.fetchone()

    return row

http://docs.djangoproject.com/en/dev/topics/db/sql/#executing-custom-sql-directly