Question

Say I have a million records in a model in the appengine datstore. Will I pay a performance penalty with the following query.

records = MyModel.all().filter("words =", "foo").fetch(offset=250000, limit=20)

From the appengine docs it says

The query has performance characteristics that correspond linearly
with the offset amount plus the limit.

Or Would I have to create an index and do something like

records = MyModel.all().filter("words =", "foo").filter("pub_date >", last_date).fetch(20)

I’m trying to see if I can query against a StringListProperty without adding any indexes to the model.

Answer 1

Using the index will almost certainly give you better performance, particularly as the number of entities increase.

If you specify an offset the datastore will scan over offset entities before your application will start getting results. The two bullet points above the one you quote explain a bit more:

The first offset results are not
skipped by the datastore itself.