Question

I have an app that has a ConcurrentQueue of items that have an ID property and a ConcurrentQueue of tasks for each item, the queue items look like:

class QueueItem {
  public int ID { get; set; }
  public ConcurrentQueue<WorkItem> workItemQueue { get; set; }
}

and the queue itself looks like:

ConcurrentQueue<QueueItem> itemQueue;

I have one thread doing a foreach over the itemQueue, deQueueing an item from each queue and doing work on it:

foreach(var queueItem in itemQueue) {
  WorkItem workItem;
  if (queueItem.workItemQueue.TryDequeue(out workItem))
    doWork(workItem);
  else
    // no more workItems for this queueItem
}

I’m using ConcurrentQueues because I have a separate thread potentially adding queueItems to the itemQueue, and adding workItems to each workItemQueue.

My problem comes when I have no more workItems in a queueItem – I’d like to remove that queueItem from the itemQueue – something like…

  if (queueItem.workItemQueue.TryDequeue(out workItem))
    doWork(workItem);
  else
    itemQueue.TryRemove(queueItem);

…but I can’t find a way to do that easily. The way i’ve come up with is to dequeue each QueueItem and then Enqueue it if there’s still WorkItems in the workItemQueue:

for (int i = 0; i < itemQueue.Count; i++) {
  QueueItem item;
  itemQueue.TryDequeue(out queueItem);
  if (queueItem.workItemQueue.TryDequeue(out workItem)) {
    itemQueue.Enqueue(queueItem);
    doWork(workItem);
  }
  else
    break;
}

Is there a better way to accomplish what I want using the PFX ConcurrentQueue, or is this a reasonable way to do this, should I use a custom concurrent queue/list implementation or am I missing something?

Answer 1

In general, there is no efficient ways to remove specific items from queues. They generally have O(1) queue and dequeues, but O(n) removes, which is what your implementation does.

One alternative structure is something called a LinkedHashMap. Have a look at the Java implementation if you are interested.

It is essentially a Hash table and a linked list, which allows O(1) queue, dequeue and remove.

This isn’t implemented in .Net yet, but there are a few implementations floating around the web.

Now, the question is, why is itemQueue a queue? From your code samples, you never enqueue or dequeue anything from it (except to navigate around the Remove problem). I have a suspicion that your problem could be simplified if a more suitable data structure is used. Could you give examples on what other pieces of code access itemQueue?