Question

Django Haystack requires the execution of

update_index

to update the solr index with new data so I’ve been wondering if there is a way to update/change only one field.

This one field must have the newest data – since it’s a number of likes. Is there a way to update the value of the likes in sqs dictionary using the newest value retrieved from the database. I’ve made a script for now but it takes way too long to execute:

sqs = SearchQuerySet().all()
d = Data.objects.all()

l = []

for i in d:
    l.append((i.id, i.like))

for i in sqs:
    for j,k in d:
        if i.id == j:
           i.like = k

Maybe there is way to speed this up?

Thanks!!

Answer 1

Here’s a simple optimization using dictionaries ( lookup complexity O(1) instead of O(n) lookup complexity for arrays ), .only and .filter methods:

d = Data.objects.all().only( "id", "like" )
d = dict( ( o.id, o.like ) for o in d )

sqs = SearchQuerySet().filter( id__in = d.keys( ) ).only( "id", "like" )

for i in sqs:
    i.like = d[ i.id ]

Note that it is more memory consuming, so make sure that queries are not too big ( split this job in multiple jobs if necessary ).

In order to obtain even bigger performance you should write direct SQL script and run it on database.