Question

I’m writing a small program to record reading progress, the data models are simple:

class BookState(db.Model):
    isbn  = db.StringProperty()
    title = db.StringProperty(required=True)
    pages = db.IntegerProperty(required=True)
    img   = db.StringProperty()

class UpdatePoint(db.Model):
    book = db.ReferenceProperty(BookState)
    date = db.DateProperty(required=True)
    page = db.IntegerProperty(required=True)

The UpdatePoint class records how many pages the user has read on corresponding date. Now I want to draw a chart from the data stored in App Engine database, the function looks like this:

book = db.get(bookkey)
ups = book.updatepoint_set
ups.order('date')

for (i, up) in enumerate(ups):
    if i == 0: continue

    # code begin
    days = (up.date - ups[i-1].date).days
    pages = up.page - ups[i-1].page
    # code end

    # blah blah

I find that for a book with about 40 update points, it will costs more than 4 seconds to run the code. And after timing I find the commented code snippet seems to be the root of poor performance. Each loop costs about 0.08 seconds or more.

It seems UpdatePoint is fetched in a lazy way that it won’t be loaded until it is needed. I want to know whether there is any better solution to accelerate the data access like fetch the data in a bunch.

Many thanks for your reply.

Answer 1

It seems I used Query class in a wrong way. I need to call ups.fetch() first to get the data. Now the code is a lot faster than before:

book = db.get(bookkey)
q = book.updatepoint_set
q.order('date')
ups = q.fetch(50)