Question

I am trying to build a generic producer/ consumer with BlockingQueue.
i want it to be as multi threaded or parallel as possible yet not eating all of the computer resources.
let say we have one producer, is it better to have consumers as threads or one consumer with
Tasks when consuming the data ?

while(true)  
{
   queue.TryTake(...) { Task.Factory.StartNew(...); }
}

or

Thread t = new Thread(Consumer.Start);

Answer 1

Maybe a combination.

Your first proposal has the problem that when the Queue is well filled you will start a lot of tasks, maybe too many.

The second one would use only 1 Thread (I assume that’s what you mean here).

You should probably start N consumers and device some strategy for N. It depends very much on the amount of work, use of I/O etc.

Possible strategies are

• N = NumberOfCores (- 1)
• Start a new Consumer when Queue.Count > M

And you might as well use a Task (with LongRunning option) instead of a Thread.