Question

I want create a cache so that when an item doesn’t exist, only one person who requested that item spends the time generating it, and others that request it at the same time will simply block until the result is cached by the first person. Here is a description of that scenario:

• Thread 1 comes in and requests the cached data for DateA
• Thread 1 sees that it isn’t in the cache and starts the relatively long
  process of generating the information
• Thread 2 comes in a sees that the information is currently being generated, waits on some kind of lock
• Thread 3 also comes in and sees that the information is currently being generated, waits on some kind of lock
• Thread 4 comes in and requests data for a different key that is already in the cache and doesn’t wait at all
• Thread 1 finishes the generation and updates the cache with the value
• Both thread 2 and 3 wake up and get the result that is now in the cache

I was thinking I would have the cache in a ConcurrentDictionary with a DateTime storing just a date as the key like ConcurrentDictionary<DateTime, CustomImmutableObject> _cache;

But I can’t see how Thread 2 and 3 would be able to wait on something. Even if there was another ConcurrentDictionary that stored some kind of status flag, how would they know when Thread 1 finishes?

Does anyone have any suggestions on how to approach developing such a cache?

Answer 1

You could use ConcurrentDictionary<DateTime, Lazy<CustomImmutableObject>> with getting the object as follows:

myDictionary
    .GetOrAdd(
        someDateTime,
        dt => new Lazy<CustomImmutableObject>(
            () => CreateMyCustomImmutableObject()
        )
    ).Value

The guarantee made by Lazy<T> in (the default) thread-safe mode will ensure that initialization occurs only once and subsequent accesses prior to the value actually being instantiated will block.

In multithreaded scenarios, the first thread to access the Value property of a thread-safe Lazy(Of T) object initializes it for all subsequent accesses on all threads, and all threads share the same data. Therefore, it does not matter which thread initializes the object, and race conditions are benign.

See here for the details.

Below is a test I wrote to ensure that I’m not spouting. This should persuade you that all is good:

void Main()
{
    var now=DateTime.UtcNow;
    var d=new ConcurrentDictionary<DateTime, Lazy<CustomImmutableObject>>();
    Action f=()=>{
        var val=d
            .GetOrAdd(
                now,
                dt => new Lazy<CustomImmutableObject>(
                    () => new CustomImmutableObject()
                )
            ).Value;
        Console.WriteLine(val);
    };
    for(int i=0;i<10;++i)
    {
        (new Thread(()=>f())).Start();
    }
    Thread.Sleep(15000);
    Console.WriteLine("Finished");
}

class CustomImmutableObject
{
    public CustomImmutableObject()
    {
        Console.WriteLine("CREATING");
        Thread.Sleep(10000);
    }
}