I want to implement a Producer – Consumers pattern using a ThreadPool for the Consumers. I will have 1 producer of requests and multiple consumers that will handle the incoming requests. When Implement the consumers using a threadpool my question if I should still have my own Queue for the producer to put requests on and then pass them to the Cosumers ThreadPool or if I should just have the Producer pass it straight to the ThreadPool Queue?
My concern with the last is how many tasks one can pass to a ThreadPool queue and at what rate? The Producer should be fairly fast in doing some ‘pre-processing’ work before passing it on to the consumer threads.
Will I not have more control when I have a Queue in between Producer – Consumer thread?
This is for a Server application that needs to be high performance and will need to handle lots of incoming client requests. (In the hundreds at a time).
Any advice is appreciated!
A queue between producer and threadpool requires 1 or 2 extra context switches: the threadpool waits on an empty queue and then needs to dispatch to a consumer thread. At the end the consumer must be presented back to the threadpool. Dispatching and handling the end can be managed by the message class.
With all consumer threads waiting on a queue, one of the listeners will consume and stop listening until ready. When all consumers are busy, the next message will be handled when the first consumer is ready.
So the former has a bit more predictable behaviour with the cost of extra contextswitches.