Question

Consider a tier of N-many subscribers, all connected to a direct exchange using identical queue name and routing key values. This creates a load-balanced system where an inbound message is send round-robin to 1 of the subscribers. This works very well for dealing with scale-out issues as more subscribers can be added as load increases and can later be withdrawn if necessary.

Now consider the requirement of being able to send messages to ALL subscribers in that tier, without knowing how many there are (for example a “reset your state” or “shutdown now please” administrative message). Is there any way to do this in rabbitmq? If this isn’t possible, is there a better approach?

My environment is Python using amqplib.

Answer 1

If I understand correctly, this is your setup:

• you have a producer publishing to a direct exchange;
• you have a queue bound to that exchange;
• you have many subscribers, all consuming from the above queue.

This works perfectly for sending a message to an arbitrary subscriber (thus sort-of load balancing), but you want to be able to send some messages to ALL the subscribers.

You could do this with a fanout exchange and an extra queue for each subscriber:

• your producer publishes system-wide messages to a fanout exchange;
• there’s a queue for each subscriber bound to that exchange;
• each subscriber first attempts to consume from this queue; if it doesn’t succeed, it tries to consume from the common queue you currently use.