Question

I’ve read many posts and blogs at this point and I’m still not sure about how to cluster correctly my 2 RabbitMQ nodes.

I’ve read the RabbitMQ clustering guide: http://www.rabbitmq.com/clustering.html

I found out about a mysterious ClusterId in the API guide, with no explanation on how to get that Id in the first place: http://www.rabbitmq.com/releases/rabbitmq-dotnet-client/v2.4.1/rabbitmq-dotnet-client-2.4.1-api-guide.pdf

Learned in that StackOverflow post that basically I’d need my clients to be aware of each node in the cluster and code for failover scenario: rabbitmq HA cluster

Now… The behavior I’d like to have is something a little more transparent if possible. Where I would potentially use that "ClusterId" on the client to make the consumer cluster aware and then hopefully the library knows to randomly connect to either node to grab messages.

Granted I know a message can only be on one server at a time, so I’m hoping for some round robin magic from the DotNet client library that would also handle fail over situations.

What I was hoping also from a publisher perspective is that the exchange would round robin distribute messages to the various nodes in the cluster. The exchange would also be cluster aware and handle fail over situations gracefully.

Now based on my readings it doesn’t quite work like that… unless I missed something.
If my knowledge is up to date and I have to code all that cluster aware business, then… why does RabbitMQ have a cluster feature in the first place? How is it used?

Is there a way to have that kind of behavior out of RabbitMQ without coding that much?

Thanks

Answer 1

I’m using HAProxy for failover and load balancing between cluster nodes for rabbitmq, also on .net client side on exception when node fails you need to manually reconnect the client.

Here is the HAProxy configuration for two node cluster, nodes are running on 15672 and 25672 ports.
Clients connect on 5672 port.

global
 daemon
 log 127.0.0.1 alert
 log 127.0.0.1 alert debug

defaults
 log global
 mode http
 option  dontlognull
 option  redispatch
 retries    3
 contimeout 5000
 clitimeout 50000
 srvtimeout 50000

listen rabbitmq 0.0.0.0:5672
  mode tcp
  balance roundrobin
  option  tcpka

server rabbit01 127.0.0.1:25672 check inter 5000 downinter 500

server rabbit02 127.0.0.1:15672 check inter 5000 backup

cudos to this blog post http://www.joshdevins.net/2010/04/16/rabbitmq-ha-testing-with-haproxy/