I started develop a distributed application and we have a big dilemma! We need a communication library easy to use and with the following features:
- reliable communication
- groups/channels
- unicast
- multicast
- auto-discover
- security
- supported by multiple programming languages
- supported on multiple operating systems
- supporting geographical distribution is a plus
A few words about this application
The main idea behind this distributed project is to have a frontend, a backend (processing application) and hundreds of agents which will collect data from some datacenters. The communication between agents and backend should be a simple as possible, we should have possibility to push configuration or to pull real time data from agents. A minimal security of the transport between agents and backend is required. Also we should have possibility to add/delete new nodes in backend cluster live. If the number of nodes of backend cluster is changing, the agents should redistribute their load (probably this should be made by backend through filtering). Usually the messages (between agents and backend) will not exceed 1500 bytes, but from time to time we should be able to send large packets (a good fragmentation management is a must).
What we found until now:
- jGroups
- Spread
- openpgm
- zeromq
- Hazelcast
If you have experience in developing clustering applications, please let me know which of the above frameworks are suitable for my project? Also, if you know other free libraries which are suitable for my project and are not listed above please let me know!
I highly recommend ZeroMQ. Also have a look at Heartbeat, the cluster messaging layer in Linux-HA. http://linux-ha.org/wiki/Heartbeat