Question

I am looking to build an Akka-based application in the cloud, for a garage startup that I’m bootstrapping; by the nature of the app, it’s semi-stateful, with as much as possible cached in RAM for performance. (It’ll be tolerant of being shut down and restarted periodically, but we want to mostly operate via cached information inside the Actors.)

The architecture is designed for a cluster of servers, communicating between them as necessary so that a user session on node A can query a middleware Actor on node B when appropriate. So my question is, how hard is that in CloudBees?

My understand from this page is that there is no automatic directory service to manage this sort of intra-cluster communication yet, but I can probably live with that — worse comes to worst, I should be able to manage discovery via the DB, with each node registering itself when it comes up and opening up many-to-many communications with the others.

What I want to check, though, is that this communication is straightforward. Does each node have a reliable local IP that it can advertise for others to contact it on, that is at least stable during this run of the application? Or is there another/better way for a node to advertise its address to the rest of the nodes running this app?

(I assume that the nodes of an app all share the same DB instance.)

Any guidance here would be greatly appreciated. I’d like to choose a hosting provider soon, and keep returning to CloudBees as the most promising-looking of the options…

Answer 1

There are no limitations currently on instances communicating with each other – the trick is in discovering membership. There is an api that will be shortly be released that will allow you to track membership – but for now, the following may work:

• To get the port, look at the file names in $PWD/.genapp/ports (as applications can have multiple ports) – (eg System.getenv(“PWD”) + “.genapp/ports” – list the files in that directory – generally will be just 1 – the file name is the port). There are other ways – for example the “sun.java.command” system property on JVM apps too.
• The hostname can be obtained via the usual means (eg InetAddress.getLocalHost().getHostName()): this host
  name will be the private name – ie it will resolve to a private IP –
  good for node to node communication.
• Public IP/hostname: perform a HTTP get (from the server) to the following URL:
  http://instance-data/latest/meta-data/public-hostname (will only
  return the public IP on the server side of course).

(see http://developer-blog.cloudbees.com/2012/11/finding-port-or-address-of-your.html)

You can then, as you say, on startup, register the appropriate port/private hostname with a DB, and then read that on each node to “seed” the cluster (akka doesn’t have to know about all members – just enough seeds) I would think a 2 phase startup: 1: register host/port, 2, look for other members, add them as seed members to the local Akka configuration (may need to periodically do the same for a while, as other nodes startup – to ensure it is seeded enough)

From my reading of Akka setup here: http://doc.akka.io/docs/akka/snapshot/scala/remoting.html

It looks like you can specify the port – so if possible, I would set that to be the app_port environment variable – that means each node can communicate via the private hostname with that port. However, http traffic will also be routed to it – can akka handle this as well – or does it need to have a discrete port for akka and another for any http interface?