Question

I have an architecture that reads packets from a packetized binary file, assigns each packet to an individual processing pipeline based on the packet type, and reassembles the packets into a new file on the other side of the pipelines. Each pipeline contains blocking queues similar to this one.

There is a thread on each side of the blocking queue in each pipline that runs a loop that queues or dequeues packets. These threads are started asynchronously (i.e. “fire and forget” style) from a controller object. This controller object has a Dictionary<int, ChannelPipeline> collection that contains all of the pipeline objects.

Here’s my question: What mechanism can I put in place that will tell me when all of the pipelines have completed processing? There’s an EndOfData property on each pipeline; do I have to continuously poll that property on every pipeline until they all read true, or is there a better (i.e. more efficient) way?

Answer 1

If you know the total number of pipelines you can consider AutoResetEvent + integer counter

AutoResetEvent allThreadsDone = new AutoResetEvent(false);
int completedThreads;
int scheduledThreads;

basically each worker thread will increment it’s value used Interlocked.Increment() and set event in case when it is last thread:

Interlocked.Increment(ref completedThreads);
if (completedThreads == scheduledThreads)
{
    allThreadsDone.Set();
}

In monitoring thread you just do:

 allThreadsDone.WaitOne();
   // here is we know that all threads are finished