I have a question that I cant seem to find an answer on.
This is my first time using RabbitMQ in a “big” application and am wondering how the queues work in a cluster.
I understand that the routing information (queues, exchanges, bindings) is on all nodes in the cluster, but the queue its self resides on the machine that it was created on. I’m creating a few durable persistent queues. If I write persistent messages to a queue on MachineA in the cluster, they get written to disk, then I write messages to MachineB, the same queue, will they be “redirected” to MachineA or will they get written to MachineB’s disks?
My concern is I will be handling thousands of messages per sec and dont want to have the speed of disks be a bottle neck. If it does in fact redirect the messages some how internally, I will have to implement some type of sharding which sucks 🙁
Let me know guys 🙂
As you say, within a RabbitMQ cluster, a queue resides only on the node it was declared on.
When you publish to that queue on a different node, the message is routed to the correct machine, where it eventually gets written to disk.
The key word here is eventually. Any message (whether published persistent or transient, on a durable or non-durable queue) may get persisted to disk. A persistent message published to a durable queue is just a strong hint to the broker that the message should get written to disk ASAP. That said, there’s no guarantee that the message will be written, so the disk will not normally become a bottleneck (the exception being if you’re running low on memory).
If you want guaranteed delivery, that’s a different question.