Question

I am writing a client in Python 2.7.2 using httplib to fetch urls from the web:

def connectHttp(url, sub_url)
    conn = httplib.HTTPConnection(url)
    try:
        conn.request("GET", "/" + sub_url)
    except Exception as ex:
        conn.close()
        logMessage('Connection problems:')
        logMessage(str(ex))
        logMessage('Exception message:')
        logMessage(ex.message)
        logMessage('-------')
        return (503, "")
    response = conn.getresponse()
    status = response.status
    data = response.read()
    conn.close()
    return (status, data)

My problem is that I don’t know how to handle redirects and I’m not even sure if httplib has an option for that. Another thing is there an easy way to prevent caching on the server (correction: proxy) (for ex, by adding a dummy query string with random data)

Much appreciated…

Answer 1

httplib2 handles redirects. Cache control is a much larger topic; the first question is whether it’s a good idea to try to force non-cached pages rather than leaving it up to the server – if the author included caching, I’d start by assuming they did so for a reason.