Question

Question 1

I am building/searching for a RAM memory cache layer for my server. It is a simple LRU cache that needs to handle concurrent requests (both Gets an Sets).

I have found https://github.com/pmylund/go-cache claiming to be thread safe.

This is true as far as getting the stored interface. But if multiple goroutines requests the same data, they are all retrieving a pointer (stored in the interface) to the same block of memory. If any goroutine changes the data, this is no longer very safe.

Are there any cache-packages out there that tackles this problem?

---

Question 1.1

If the answer to Question 1 is No, then what would be the suggested solution?
I see two options:

Alternative 1
Solution: Storing the values in a wrapping struct with a sync.Mutex so that each goroutine needs to lock the data before reading/writing to it.
type cacheElement struct { value interface{}, lock sync.Mutex }
Drawbacks: The cache becomes unaware of changes made to data or might even have dropped it out of the cache. One goroutine might also lock others.

Alternative 2
Solution: Make a copy of the data (assuming the data in itself doesn’t contain pointers)
Drawbacks: Memory allocation every time a cache Get is performed, more garbage collection.

---

Sorry for the multipart question. But you don’t have to answer all of them. If you have a good answer to Question 1, that would be sufficient for me!

Answer 1

Alternative 2 sounds good to me, but please note that you do not have to copy the data for each cache.Get(). As long as your data can be considered immutable, you can access it with many multiple readers at once.

You only have to create a copy if you intend to modify it. This idiom is called COW (copy on write) and is quite common in concurrent software design. It’s especially well suited for scenarios with a high read/write ratio (just like a cache).

So, whenever you want to modify a cached entry, you basically have to:

1. create a copy of the old cached data, if any.
2. modify the data (after this step, the data should be considered immutable and must not be changed anymore)
3. add / replace the existing element in the cache. You could either use the go-cache library you have pointed out earlier (which is based on locks) for that, or write your own lock-free library that simply swaps the pointers to the data element atomically.

At this point any goroutine that performs a cache.Get operation will get the new data. Existing goroutines however, might still be reading the old data. So, your program might operate on many different versions of the same data at once. But don’t worry, as soon as all goroutines have finished accessing the old data, the GC will collect it automatically.