Question

I’m using Ruby’s EventMachine library for accessing external RESTful APIs asynchronously from a Ruby server I’m writing. One of the methods I make available through my server requires that multiple external APIs be accessed and the responses aggregated before returning a response to the client accessing my server. Here’s how I’m doing it now…

def aggregate
  responses = Array.new
  done      = 0

  # clients are EM::Deferrable objects...
  clients.each do |client|
    client.callback do |response|
      responses << response
      done += 1
    end

    client.errback do |response|
      done += 1
    end
  end

  until done == clients.length
    sleep 1
  end

  return responses
end

Are there better ways of handling this type of situation? Is it possible that the EM::Completion class being shipped in version 1.0.0 of EventMachine will provide plumbing that better supports such a use case?

I’m sure this issue has come up before (although I couldn’t find a similar question on StackOverflow), so I’m curious to see how others have approached the problem.

Answer 1

First of all, using sleep entirely defeats the purpose of EventMachine. You’re blocking the event loop with it. I’d need a bigger picture for a suggestion on how to do it better, but your code should get used to the fact that a method returns before all requests are done. Alternatively look at em-synchrony for how to do procedural style evented programming with Ruby’s fibers (transparent to you).

One way to reduce the glue code to run multiple requests in parallel is to use something like em-http-requests multi interface, which works with both EventMachine’s callback way, and em-synchrony too.

EM::Completion on the other hand is more of an extended Deferrable, won’t exactly help you in this case.