Question

I was expecting postgres to take some time, but wasn’t expecting for it to not be the bottleneck here, how can i solve this?

2772856 function calls (2578490 primitive calls) in 32.361 seconds

Ordered by: internal time

ncalls  tottime  percall  cumtime  percall filename:lineno(function)
127921/19340    3.670    0.000   11.486    0.001 /usr/lib/python2.7/copy.py:145(deepcopy)
 2240    3.031    0.001    3.244    0.001 {method 'execute' of 'psycopg2._psycopg.cursor' objects}
114402    1.541    0.000    2.348    0.000 /usr/lib/python2.7/copy.py:267(_keep_alive)

———————————————————————————-

    suppliers = models.Supplier.objects.all().order_by('company')
    for supplier in suppliers :
        sup = {}
        sup['company'] = supplier.company
        sup['supplies'] = get_supplies(1, supplier.uuid)
        sup['category'] = 'Supplier'
        if isocode == None :
            addresses = models.Address.objects.filter(company = supplier.company)
        else :
            addresses = models.Address.objects.filter(company = supplier.company, country_iso = isocode)
        sup['contacts'] = models.Contact.objects.filter(address__in=addresses)
        company_list.append(sup)

———————————————————————————-

def get_supplies (bought_in_controlpanel_id, supplier_uuid) :

    supplier    = None
    activenode  = None

    if supplier_uuid is not None :
        supplier = models.Supplier.objects.get(uuid = supplier_uuid)

    try :
        activenode = boughtin.BoughtInControlPanel.objects.get (pk = 1)
    except :
        pass

    supplies = boughtin.BoughtInControlPanel.objects.filter (parent = activenode)
    for supply in supplies :
        supply.checked  = 0
        supply.disabled = ""
        supply.open     = 0

        if supplier_uuid is not None :
            try    :
                models.Supplies.objects.get(supplier = supplier, bought_in_control_panel = supply)
                supply.checked = 1
            except :
                supply.open    = 1

    return supplies

Answer 1

Performance is usually bad when you query DB in a loop. Try avoiding that.

Why not continue using relations and __in query? =)

I think this should work with less SQL queries for the first example:

company_list = models.Supplier.objects.values_list('company', flat=True)

filter_kwargs = dict(address__company__in=company_list)

if isocode is not None:
    filter_kwargs.update(dict(address__company__isocode=isocode))

sup['contacts'] = models.Contact.objects.filter(**filter_kwargs)

Regarding get_supplies(), I’d avoid Supplies.objects.get() in loop as well, unless you’re sure it will be hit just a couple of times. It may be better to gather a list of supplies with one larger query before the loop, and then simply check if required item is present in that list. Although you should profile both variants and select the faster one.