Question

I have a shared memory pool from which many different threads may request an allocation. Requesting an allocation from this will occur a LOT in every thread, however the amount of threads is likely to be small, often with only 1 thread running. I am not sure which of the following ways to handle this are better.

Ultimately I may need to implement both and see which produces more favorable results… I also fear that even thinking of #2 may be premature optimization at this point as I don’t actually have the code that uses this shared resource written yet. But the problem is so darn interesting that it continues to distract me from the other work.

1) Create a mutex and have a thread attempt to lock it before obtaining the allocation, then unlocking it.

2) Have each thread register a request slot, when it needs an allocation it puts the request in the slot, then blocks(while (result == NULL) { usleep() }) waiting for the request slot to have a result. A single thread continuously iterates request slots making the allocations and assigning them to the result in the request slot.

Number 1 is the simple solution, but a single thread could potentially hog the lock if the timing is right. The second is more complex, but ensures fairness among threads when pulling from the resource. However it still blocks the requesting threads, and if there are many threads the iteration could burn cycles without doing any actual allocations until it finds a request to fulfill.

NOTE: C on Linux using pthreads

Answer 1

Solution 2 is bogus. It’s an ugly hack and it does not ensure memory synchronization.

I would say go with solution 1, but I’m a little bit skeptical of the fact that you mentioned “memory pool” to begin with. Are you just trying to allocate memory, or is there some other resource you’re managing (e.g. slots in some special kind of memory, memory-mapped file, textures in video memory, etc.)?

If you are just allocating memory, then you’re completely right to be worried about premature optimization. The whole problem is premature optimization, and the system malloc will do as well as or better than your memory pool will do. (Or if your code will be running on one of the few systems with a pathologically broken malloc like some video game consoles, just drop in a replacement only on those known-broken systems.)

If you really do have a special resource you need to manage, start with solution 1 and see how it works. If you have problems, you might find you can improve it with a condition variable where the resource manager notifies you when a slot can be allocated, but I really doubt this will be necessary.