Question

I’m creating a memoization cache with the following characteristics:

• a cache miss will result in computing and storing an entry
  • this computation is very expensive
  • this computation is idempotent
• unbounded (entries never removed) since:
  • the inputs would result in at most 500 entries
  • each stored entry is very small
  • cache is relatively shorted-lived (typically less than an hour)
  • overall, memory usage isn’t an issue
• there will be thousands of reads – over the cache’s lifetime, I expect 99.9%+ cache hits
• must be thread-safe

What would have superior performance, or under what conditions would one solution be favored over the other?

ThreadLocal HashMap:

class MyCache {
    private static class LocalMyCache {
        final Map<K,V> map = new HashMap<K,V>();

        V get(K key) {
            V val = map.get(key);
            if (val == null) {
                val = computeVal(key);
                map.put(key, val);
            }
            return val;
        }
    }

    private final ThreadLocal<LocalMyCache> localCaches = new ThreadLocal<LocalMyCache>() {
        protected LocalMyCache initialValue() {
            return new LocalMyCache();
        }
    };

    public V get(K key) {
        return localCaches.get().get(key);
    }
}

ConcurrentHashMap:

class MyCache {
    private final ConcurrentHashMap<K,V> map = new ConcurrentHashMap<K,V>();

    public V get(K key) {
        V val = map.get(key);
        if (val == null) {
            val = computeVal(key);
            map.put(key, val);
        }
        return val;
    }
}

I figure the ThreadLocal solution would initially be slower if there a lot of threads because of all the cache misses per thread, but over thousands of reads, the amortized cost would be lower than the ConcurrentHashMap solution. Is my intuition correct?

Or is there an even better solution?

Answer 1

use ThreadLocal as cache is a not good practice

In most containers, threads are reused via thread pools and thus are never gc. this would lead something wired

use ConcurrentHashMap you have to manage it in order to prevent mem leak

if you insist, i suggest using week or soft ref and evict after rich maxsize

if you are finding a in mem cache solution ( do not reinventing the wheel )
try guava cache
http://docs.guava-libraries.googlecode.com/git/javadoc/com/google/common/cache/CacheBuilder.html