Question

I am importing records from XML files, and it is chewing up memory. I’ve gone through the XML ElementTree that is created and deleted all of the elements and subelements to clear memory, but it still just piles up.

I finally found a solution, and wanted to have this in SO for others who may have the same problem in the future. I was scouring the web for answers, when I stumbled upon this: Django MemoryError – How to work with large databases. My MemoryError is not because of the size of my Queryset, because those aren’t that big, but I figured I’d give it a try anyway. I changed all of my objects.all() to objects.all().iterator() for any loops over objects. That didn’t do much, as I was only looping over about three different sets of objects, none of them huge sets. I reluctantly turned DEBUG=True to DEBUG=False in settings.py and it cut down the memory usage to almost nothing! From the Django Docs:

“It is also important to remember that when running with DEBUG turned on, Django will remember every SQL query it executes. This is useful when you’re debugging, but it’ll rapidly consume memory on a production server.“

This is not a production server, it is my development server. So, since all of the queries executed are stored, where are they and how can I get them? If they are so helpful for debugging, why aren’t they more readily available?

Answer 1

Here’s how to display the queries:

from django.db import connection
print connection.queries

For more info see this faq entry.