Question

In my App Engine app (in Python 2.7, threadsafe) I want to be able to display information on all entities of my ChildModel kind. I think I’ve got memcache working, but because the ChildModel has a property that connects to a ParentModel there are some things happening that I don’t understand.

I currently have the following models:

class ParentModel(db.Model):
  name = db.StringProperty()
  # currently 109 of these

class ChildModel(db.Model):
  name = db.StringProperty()
  parent_program = db.ReferenceProperty(ParentModel)
  # currently 758 of these

I’m implemented memcache using the example from Nick Johnson’s blog.

class AllEntities(webapp2.RequestHandler):
  def get(self):
    entitylist = deserialize_entities(memcache.get("entitylist"))
    if not entitylist:
      entitylist = ChildModel.all().fetch(None)
      memcache.set("entitylist", serialize_entities(entitylist))
    totalnum = ChildModel.all().count()

The very first time I run this I see the following in appstats:

datastore_v3.Get        758
datastore_v3.RunQuery   3
datastore_v3.Next       2
memcache.Get        1
memcache.Set        1

After that, I see the following in appstats:

datastore_v3.Get        758
datastore_v3.RunQuery   2
memcache.Get            1

It looks like memcache gets set correctly, based on the memcache size (1304599 byte(s)). But I can’t figure out how to stop the 758 datastore_v3.Get’s which are extraordinary slow and also killing me on my quota of Datastore Small Operations and Datastore Read Operations.

Can someone please help me figure out what I’m doing wrong?

Answer 1

You are not doing anything wrong, count() is whats doing the 758 gets and cost you the small operation quota.

Since you have all the models loaded from memcache into a list you can just call len(entitylist) to get the amount of entities.

---

Try calling prefetch_refprops(entitylist, ChildModel.parent_program) prior to setting the value in the memcache as describe in Nick’s blog.