Question

How to prevent loading the value that is not present in the cache many times simultanously, in the efficient way?

A typical cache usage is the following pseudocode:

Object get(Object key) {
 Object value = cache.get(key);
 if (value == null) {
  value = loadFromService(key);
  cache.set(key,value);
 }
 return value;
}

The problem: before the value is loaded from service (Database, WebService, RemoteEJB or anything else) a second call may be made in the same time, which will make the value loaded once again.

For example, when I’m caching all items for user X, and this user is often viewed, and have many items, there’s high probability of calling the load of his all items simultanously, resulting in heavy load on the server.

I could make get function synchronized, but this would force other searches to wait, making not much sense. I could create new lock for every key, but I don’t know if it’s a good idea to manage such large number of locks in Java (this part is language specific, the reason I’ve tagged it as java).

Or there is another approach I could use? If so, what would be the most efficient?

Answer 1

Don’t reinvent the wheel, use guava’s LoadingCache or memoizing supplier.

If you are using Ehcache, read about read-through, this is the pattern you are asking for. You must implement the CacheEntryFactory interface to instruct the cache how to read objects on a cache miss, and you must wrap the Ehcache instance with an instance of SelfPopulatingCache.