Question

Before I reinvent the wheel, is there a topic-like concurrent queue in plain Java? I have the following requirements:

• Multiple readers/consumers
• Multiple writers/producers
• Every message must be consumed by every (active) consumer
• After every consumer reads a message it should become garbage (i.e. no more references)
• Writing to the queue should not be O(N) to the number of consumers
• Concurrent, preferably non-blocking
• Not JMS based: it’s for a much lighter/embeddable environment

That’s pretty much everything I need. Any pointers?

Answer 1

Basically you are talking about multiplexing, and no there isn’t something in the standard lib but it is pretty simple to create one. Presuming that your clients aren’t interested in messages published before they subscribe then you need a pool of queues for each consumer and publication simply offers the item to each queue:

public class Multiplexer<M> {
  private final List<BlockingQueue<M>> consumers 
    = new CopyOnWriteArrayList<BlockingQueue<M>>();

  public void publish(M msg) {
    for (BlockingQueue<M> q : consumers) {
      q.offer(msg);
    }
  }

  public void addConsumer(BlockingQueue<M> consumer) {
    consumers.add(consumer);
  }
}

This version allows consumers to use whatever blocking queue implementation they might want. You could obviously provide a standard implementation and a nice interface for the client if you want.