Question

In my web application, I have several server modules which respond to client modules through RESTful services.

Let me give a use case here. Suppose I am building a social network where the instances of the server modules handle the responsibility of managing the tweets of different users.

Each of these servers handle requests for different set of data. For e.g. Server1 will serve the tweets of user U1, U2 and U3, Server2 for user U4 and U5, Server3 for user U6, U7 and U8. The implementation of these servers would be same, only the user data that they handle differs. This configuration might even change at runtime i.e. the responsibility to handle data of a particular user’s tweets can change from one server to another.

Now suppose a client module wants data for user U3. How shall it determine (at runtime) which server it has to contact.

Basically, what are the various ways to implement this kind of dynamic service registration and discovery? This being a realtime application, it is desirable to cache the service discovery results by the client module to save the networking hops for service discovery on each request.

Are there some existing open source solutions which can help me serve this functionality or should I go ahead and implement this myself?

Answer 1

This looks like the responsibility for a Reverse Proxy.

Varnish is one case. I’d suggest you combine w/ it, as it already does a great part of your needs.