Question

For my current project, we’ve decided to deploy our application to Amazon’s Elastic Computing Cloud on some Linux boxes. We use JGroups for group communication and needed a reliable discovery mechanism that didn’t require preconfiguring each application with the addresses of the other cluster members (which is necessary with TCPPING, and ‘sort’ of necessary with TCPGOSSIP). Since we can’t use UDP multicast, that excludes multicast discovery from our options.

We looked into using the S3 Ping protocol, but after reading that there were some reliability issues with it, we decided to roll our own protocol do accomplish this discovery.

I’d love to get some feedback on the simple protocol that we wrote and how it might compare to the S3 Ping. The one limitation it currently has is that it depends on the AWS SDK for Java.

public class EC2Ping extends Discovery {

    private static final Logger log = LoggerFactory.getLogger(EC2Ping.class);

    public static final short EC2_PING_PROTOCOL_ID = 1001;
    private static final int DEFAULT_JGROUPS_PORT = 7800;

    static {
        ClassConfigurator.addProtocol(EC2_PING_PROTOCOL_ID, EC2Ping.class);
    }

    /** The JGroups port number */
    private int port = DEFAULT_JGROUPS_PORT;

    /** The EC2 client */
    private AmazonEC2Client client;

    /** The EC2 instance filters */
    private List<Filter> filters;

    public EC2Ping(EC2Ping src) {
        this.client = src.client;
        this.port = src.port;
    }

    public EC2Ping() {
        // Default constructor
    }

    @Required
    public void setClient(AmazonEC2Client client) {
        this.client = client;
    }

    public void setFilters(List<Filter> filters) {
        this.filters = filters;
    }

    public void setPort(int port) {
        this.port = port;
    }

    public int getPort() {
        return port;
    }

    @Override
    public Collection<PhysicalAddress> fetchClusterMembers(String cluster_name) {
        List<PhysicalAddress> addresses = new ArrayList<PhysicalAddress>();
        DescribeInstancesRequest request = new DescribeInstancesRequest();
        if (filters != null) {
            request.setFilters(filters);
        }
        DescribeInstancesResult result = client.describeInstances(request);
        for (Reservation res : result.getReservations()) {
            for (Instance instance : res.getInstances()) {
                String ipAddr = instance.getPrivateIpAddress();
                IpAddress addr;
                try {
                    addr = new IpAddress(ipAddr, port);
                    addresses.add(addr);
                } catch (UnknownHostException uhe) {
                    log.error("Unable to resolve cluster member address [" + ipAddr + "]");
                }
            }
        }
        return addresses;
    }

    @Override
    public boolean isDynamic() {
        return true;
    }

    @Override
    public boolean sendDiscoveryRequestsInParallel() {
        return true;
    }
}

I can include my protocol stack configuration if necessary, but it is very similar to UDP except that instead of Multicast discovery, it uses our EC2Ping protocol.

My main questions are as follows:

1. Does this present a more reliable solution than the S3 Ping?
2. Does the dependency on the AWS Java SDK negate the usefulness of this solution? (in terms of contributing back to JGroups)

Any comments would be greatly appreciated. Thanks

Answer 1

You may want to have a look at the JGroups AWS Project up on GitHub. It uses the AWS APIs to build clusters using EC2 tags. It also supports instance profiles, so you can get the access key and secret key out of your configuration file.