Question

Hi I create three RabbitMQ servers running in cluster on EC2

I want to scale out RabbitMQ cluster base on CPU utilization but when I publish message only one server utilizes CPU and other RabbitMQ-server not utilize CPU

so how can i distribute the load across the RabbitMQ cluster

Answer 1

RabbitMQ clusters are designed to improve scalability, but the system is not completely automatic.

When you declare a queue on a node in a cluster, the queue is only created on that one node. So, if you have one queue, regardless to which node you publish, the message will end up on the node where the queue resides.

To properly use RabbitMQ clusters, you need to make sure you do the following things:

• have multiple queues distributed across the nodes, such that work is distributed somewhat evenly,
• connect your clients to different nodes (otherwise, you might end up funneling all messages through one node), and
• if you can, try to have publishers/consumers connect to the node which holds the queue they’re using (in order to minimize message transfers within the cluster).

Alternatively, have a look at High Availability Queues. They’re like normal queues, but the queue contents are mirrored across several nodes. So, in your case, you would publish to one node, RabbitMQ will mirror the publishes to the other node, and consumers will be able to connect to either node without worrying about bogging down the cluster with internal transfers.