Question

Let’s say I have a multithreaded C++ program that handles requests in the form of a function call to handleRequest(string key). Each call to handleRequest occurs in a separate thread, and there are an arbitrarily large number of possible values for key.

I want the following behavior:

• Simultaneous calls to handleRequest(key) are serialized when they have the same value for key.
• Global serialization is minimized.

The body of handleRequest might look like this:

void handleRequest(string key) {     KeyLock lock(key);     // Handle the request. } 

Question: How would I implement KeyLock to get the required behavior?

A naive implementation might start off like this:

KeyLock::KeyLock(string key) {     global_lock->Lock();     internal_lock_ = global_key_map[key];     if (internal_lock_  == NULL) {         internal_lock_  = new Lock();         global_key_map[key] = internal_lock_;     }     global_lock->Unlock();     internal_lock_->Lock(); }  KeyLock::~KeyLock() {     internal_lock_->Unlock();     // Remove internal_lock_ from global_key_map iff no other threads are waiting for it. } 

…but that requires a global lock at the beginning and end of each request, and the creation of a separate Lock object for each request. If contention is high between calls to handleRequest, that might not be a problem, but it could impose a lot of overhead if contention is low.

Answer 1

You could do something similar to what you have in your question, but instead of a single global_key_map have several (probably in an array or vector) – which one is used is determined by some simple hash function on the string.

That way instead of a single global lock, you spread that out over several independent ones.

This is a pattern that is often used in memory allocators (I don’t know if the pattern has a name – it should). When a request comes in, something determines which pool the allocation will come from (usually the size of the request, but other parameters can factor in as well), then only that pool needs to be locked. If an allocation request comes in from another thread that will use a different pool, there’s no lock contention.