Question

I need to write a python script that makes multiple HTTP requests to the same site. Unless I’m wrong (and I may very well be) urllib reauthenticates for every request. For reasons I won’t go into I need to be able to authenticate once and then use that session for the rest of my requests.

I’m using python 2.3.4

Answer 1

If you want to keep the authentication you need to reuse the cookie. I’m not sure if urllib2 is available in python 2.3.4 but here is an example on how to do it:

req1 = urllib2.Request(url1)
response = urllib2.urlopen(req1)
cookie = response.headers.get('Set-Cookie')

# Use the cookie is subsequent requests
req2 = urllib2.Request(url2)
req2.add_header('cookie', cookie)
response = urllib2.urlopen(req2)