Question

How can I watch the messages being sent back and for on urllib shttp requests? If it were simple http I would just watch the socket traffic but of course that won’t work for https. Is there a debug flag I can set that will do this?

import urllib
params = urllib.urlencode({'spam': 1, 'eggs': 2, 'bacon': 0})
f = urllib.urlopen("https://example.com/cgi-bin/query", params)

Answer 1

No, there’s no debug flag to watch this.

You can use your favorite debugger. It is the easiest option. Just add a breakpoint in the urlopen function and you’re done.

Another option would be to write your own download function:

def graburl(url, **params):
    print "LOG: Going to %s with %r" % (url, params)
    params = urllib.urlencode(params)
    return urllib.urlopen(url, params)

And use it like this:

f = graburl("https://example.com/cgi-bin/query", spam=1, eggs=2, bacon=0)