Question

I need to cache some costly queries(5sec). I recently came to know about dogpile affect when using memcache. One solution is using locks on cache key-value pair to prevent this effect. Since GAE memcached doesn’t support locks on memcached key-values, what is the best practice to prevent dogpile effect?

Answer 1

It depends on what your app is doing, but here are three approaches that can be useful (none is perfect for all scenarios, they each have downsides):

If you can respond to the request with something like ‘wait a bit and try again’ then setting a flag that represents that into memcache on a miss, before you spend the time re-processing the real value could help (there will still be the possibility of a race between the get and the first set, but it will be much shorter than waiting for the query to be done):

value = memcache.get(key)
if value is None:
  memcache.set(key, 'recalculating')
  // do the slow thing
  memcache.set(key, actual_result)

return value

Alternatively, if you’re setting a lifetime on your cached results, but could cope with clients receiving slightly stale data, then as well as caching your value with a timeout and a key, you cache a copy with no timeout and a different key, then on miss use this copy to re-populate the cache while you re-process a fresh value (again, still the chance of a race between the get/set, and also it’s possible the copy will be evicted.):

value = memcache.get(key)
if value is None:
  memcache.set(key, cache.get(key+'copy'))
  // do the slow thing
  memcache.set(key, actual_result, 30)
  memcache.set(key+'copy', actual_result)

return value

The third is simpler, simply have a backend constantly doing your query and have it update the cache, so that frontend requests are less likely to have to do it. This does mean, though, that the queries are being done regardless of the values being used by anyone.