Question

I am looking for a lib that lets me roughly:

• connect to localhost:port, but see http://somesite.com
• rewrite all static assets to point to localhost:port instead of somesite.com
• support cookies / authentication

i know that http://betterinternet.co/ does this already, but they wont give me their source code for some reason.

I assume this doesnt exist as free code, so if i were to write one, are there any nuances to it? If i replace all occurrences of somesite.com in html and headers, will that be enough?

Answer 1

I don’t see any straight forward solution to your problem. If I’ve understood correctly, you want a caching HTTP proxy which serves static contents locally, with URL rewriting rules defined in Python (or nodejs). That’s quite a task.

A caching HTTP proxy implementation is not trivial. So I’d use an existing implementation, such as Squid (or Apache if it does caching too).

You could then place a (relatively) simple HTTP server written in Python in front of that (e.g. based on BaseHTTPServer and urllib2) which performs the URL rewriting as you want them and forwards the requests to the proxy (or direct to internet).

The idea would be to rely on the proxy setup to perform all the processing you don’t want to modify (including basic rewrite rules, authentication, caching and cache management) and limit your front-end implementation to performing only the custom rewriting you are interested in.