Question

I have a list of lists, like so:

li = [[('A', 'one'), ('A', 'two')], [('B', 'three'), ('B', 'four')]]

and I need to query a mongo database to get all objects whose list field contains at least ONE item in each sublist of li. Ex: items that have either [(‘A’, ‘one’) OR (‘A’, ‘two’)] AND either [(‘B’, ‘three’) OR (‘B’, ‘four’)] …

I am using mongoengine, but that may be able to change if I can use something else to get this done. So right now I am doing many queries like so, to avoid getting duplicate entries:

final = set()
for sublist in li:
    query = Obj.objects(list_field__in=sublist)
    final &= set(query)

The problem is that this is very slow when dealing with large query results (making the set takes a very long time, I believe). Is there a way that I can speed this up? Specifically, is there a way that I can avoid set/list creation from the query results?

I would really like to be able to somehow write something like this:

query = Obj.objects(list_field__in=li[0] AND list_field__in=li[1] AND ...)

Edit: The answer below does not work upon further testing, because mongoengine does not allow Q(field=x) & Q(field=y)

Edit2: Here is the equivalent mongoDB query that I want to do:

db.obj.find({ "$and": [
    {"list_field": {"$in":
        [["A", "one"], ["A", "two"]]
    }},
    {"list_field": {"$in":
        [["B", "three"], ["B", "four"]]
    }}
]})

Can I do this in mongoengine? It will not let me do a query with Q(list_field__in=[('A', 'one'), ('A', 'two')]) | Q(list_field__in=[('B', 'three'), ('B', 'four')])

Answer 1

I think you can try it through the Q class:

  filter = reduce(Q.__and__, map(lambda x: Q(list_field__in=x), li))
  Obj.objects(filter)