Question

To avoid the high latency (spikes) in GAE datastore writes, I want to implement a write-behind cache (using the Java low-level API). This means that data is written synchronously to the memcache, and then asynchronously to the datastore, so that the request can return quickly.

This, however, means that I need to somehow need to deal with Exceptions arising from datastore contentions (e.g. to initiate a retry) also asynchronously. More precisely, I need to be able to react to contention’s occurring after the request has returned. How can I do that? Using the task queue for async write processing is not an option, because pushing to the queue is said to be only marginally faster than a datastore write.

If that is impossible, then what are good ways to implement a write-behind cache? Or how to deal with slow writes in a scenario where data loss is not an option.

Answer 1

Memcache is volatile and it may flush data at any time, so this approach is very unreliable.

You would be best off using Push Task Queue. Use it via DeferredTask helper class. Here is an example.