Question

I have a Rails project and two ruby mini-daemons running in the background. What’s the best way to communicate between them?

Communication like below should be possible:
Rails -> Process 1 -> Process 2 -> Rails

Some requests would be sync, other async.

Queues (something like AMQ, or custom Redis based) or RPC HTTP calls?

Answer 1

I implemented a system via RabbitMq + the bunny gem.

Update:

After reading http://blog.brightbox.co.uk/posts/queues-and-callbacks I decided to try out RabbitMQ. There are two gems amqp (async, eventmachine based) or bunny (sync). amqp is great, but if you’re using Rails with passenger it can do some weird things.

The system works like this, the daemons listen on a queue for messages:

# The incoming data should be a JSON encoded hash that looks like:
# { "method" => method_to_call, "opts" => [ Array of opts for method ],
#   "output" => "a queue where to send the result (optional)" }
# If output is specified it will publish the JSON encoded response there.
def listen_on(queue_name, class)
  BUNNY.start
  bunny = BUNNY.queue(queue_name)

  bunny.subscribe do |msg|
    msg = JSON.parse(msg[:payload])

    result = class.new.send(msg["method"], *msg["opts"])

    if msg["output"]
      BUNNY.queue(msg["output"]).publish(result.to_json)
    end
  end

So once a message is received it calls a method from a class. One thing to note is that it would have been ideal to use bunny for Rails and amqp in the daemons. But I like to use one gem pe service.