Question

I’ve been studying .Net threading and have learned about the Threading.Task and Threading.Task.Parallel classes. Right now using the ForEach method of Parallel class I process each object on a separate thread. However I beleive the ForEach method waits for all threads to finish executing before it comes back. This results in those threads that finish before others to sit idle. I want to have these threads working constantly. So this is what I’m hoping to do:

I have thread A in charge of slice selecting records from the table and then starting n worker threads to process each record.

1. Every time a worker thread finishes its job, it should ask thread A for the next record to process.

2. When allocating records to worker threads, if thread A runs out, it should go back to the database and fetch some more. I’m guessing this allocating process has to be wrapped within a critical section block.

Can anyone point me to a tutorial that fits my scenario?

Answer 1

For a producer consumer scenario you can use the ConcurrentQueue which is already thread safe.

That means that you can fill it from any A thread, and pop out items to work on from all other consumer threads without using locking.