I learned about semaphores from an earlier question I had today, and I stll am scratching my head here. There seems to be no discussion about scope beyond global and local, where global is defined as the entire operating system.
If I have an application made from several assemblies, and each assembly has several classes, and each class has a private static semaphore object, with different “queue” lengths, if I start queuing different tasks up in my application thread pool in different places, how does that work? How do the threads behave around each other? All the examples I see include one or two classes in one assembly, and I’m not getting a clear picture on how this works.
I use thread pooling all over my app. It parallelizes data (sending customized emails to various people, generating customized reports en masse, collecting data from various web services, etc) while leaving my interface responsive, which is a wonderful thing.
One of my web service sources limits me to five concurrent connections, and I could not figure out how to limit the web requests to 5 active threads while still allowing the rest of the application to utilize other threads as necessary. So, I turned to SO, and asked how to do it. The proposed answer was use Semaphores.
Until that point, I did not know a thing about semaphores, so I researched it. It does indeed seem to limit the number of threads executing a specific method, but it does not make sense how this communicates properly with the thread pool manager. If I implemented a semaphore on my web request functionality, and I get a backlog of threads waiting to perform web service calls, how does the thread pool know (can it know?) to issue more threads for other processes? The scope of the semaphore is private; it shouldn’t see the object.
Further, is that what the semaphore is supposed to do? Can I likewise limit other groups of tasks, by having them share a common semaphore? Is this a gross bastardization of the intent of a semaphore, or exactly what it’s meant to do. There’s so much information out there on it, but in simplified, abstract form, and I couldn’t find an article describing when and how it is appropriate to use these things.
So how does a private static semaphore communicate with the thread pool so the thread pool knows whether or not to spawn another worker thread? Does it? Will I be creating more problems than solutions by doing this? What sort of behavior can I expect my thread pool to exhibit with a backlog of web requests? Will it spawn new threads for the web requests until it’s “full”, reducing the thread availability for other methods? Can I make it not do that?
The scope constraints, (if you can call them that!), are because the semaphores are OS kernel synchronization primitives that can be used, (unnamed), for inter-thread comms or, (named), or inter-process, (inter-assembly) comms. The language cannot restrict the scope of the named variant.
There is a huge pile of information on semaphores in general on Google. For .NET-wapped ones, MSDN.
Inter-process signalling and communication via. named semaphores is certainly possible in general. How you might do it in the managed environment is another matter. In unmanaged code, it usually involves other comms elements, like shared memory areas and/or memory-mapped files. You should probably not go there.
Be careful about trying to constrain thread pools in different assemblies by making the tasks signa/wait on named semaphores. By all means try it, if you think it may solve some problem with your app, but there is at least the possibility that the pool thread counts, running pool threads, pool threads blocked on semaphores, pool threads blocked inside tasks on IO etc. may become unstable.