Question

I would like to run a small http proxy server on my machine which sends all requests to an upstream / cascading proxy server.

Twisted Web HTTP Proxy -> Squid -> Internet connection

Unfortunately I cannot figure out how to send all ProxyClient requests upstream / cascade them to Squid. Could you give me a hint?

Do I have to use reactor.connectTCP and HttpClientFactory for this and somehow direct the Squid response through?

I have used the following Twisted code sample to get the twisted http proxy up:

from twisted.web import proxy, http
from twisted.internet import reactor
from twisted.python import log
import sys
log.startLogging(sys.stdout)

class ProxyFactory(http.HTTPFactory):
    protocol = proxy.Proxy

reactor.listenTCP(8080, ProxyFactory())
reactor.run()

Thanks,

Mathias

Answer 1

I think you’re over complicating things. To me, it sounds like what you want to do is round-robin load balance http proxy requests between two squid services.

If it was me, I wouldn’t write a byte of code and use something off the shelf.

I’d use HA-Proxy with a configuration something like this:

global
        daemon
        maxconn 256

defaults
        mode http
        timeout connect 5000ms
        timeout client 50000ms
        timeout server 50000ms

frontend loadbal
        bind *:3128
        default_backend squids
        option http_proxy

backend squids
        option http_proxy
        server squid1 192.168.1.2:3128
        server squid2 192.168.1.3:3128
        balance roundrobin

You might need to do some playing around, some webservices may require certain types of stickiness, but this application is highly configurable in this regard. It looks like HA-Proxy v 1.4 supports keep alive connections, which may simplify some of the corner cases I can imagine.

Oh, also, if you’re trying to get the most out of your two connections, I’d suggest that you configure ICP between your two squid boxes.