Question

I have a syncer process and it makes curl HTTPS calls to Twitter, Facebook, etc to gather data for our users and update the DB (stored in local file system). There is a HTTP server running on the same system which queries this db and returns results to the user when the request.

The problem is that the sycer process is eating up too much CPU/IO due to SSL negotiation happening while calling Twitter, Facebook, etc APIs. This is making user experience a bit slow. So I want this SSL negotiations to happen on another system from where the HTTP server is running. And I can not take the syncer process outside because it needs local file system access to update the DB.

So I was thinking of building a kind of proxy server. The syncer process will make HTTP calls to the proxy server. The proxy server will make HTTPS calls to Twitter, Facebook, etc and returns the results to the syncer process.

Is any software anything existing for this? I don’t want to reinvent the wheel if something is already there.

Answer 1

I ended up writing a simple proxy server using nodejs. Clients which need to call HTTPS apis will pass all the details in JSON format to this proxy server. This proxy server will in turn call HTTPS apis and return the result to clients.