Question

I’m trying to learn Python. At the moment I’m stuck with a simple thing 🙁
When I send request with urllib2, it add “Connection:close” header. What should I do to send request WITHOUT this header?

My code:

request = urllib2.Request('http://example.com')
request.add_header('User-Agent', self.client_list[self.client])
opener = urllib2.build_opener()
opener.open(request).read()

Thanks guys, I’m very appreciate your help!

Answer 1

You can’t. Not with urllib2 in any case. From the source code:

# We want to make an HTTP/1.1 request, but the addinfourl
# class isn't prepared to deal with a persistent connection.
# It will try to read all remaining data from the socket,
# which will block while the server waits for the next request.
# So make sure the connection gets closed after the (only)
# request.
headers["Connection"] = "close"

Use the requests library instead, it supports connection keep-alive:

>>> import requests
>>> import pprint
>>> r = requests.get('http://httpbin.org/get')
>>> pprint.pprint(r.json)
{u'args': {},
 u'headers': {u'Accept': u'*/*',
              u'Accept-Encoding': u'gzip, deflate, compress',
              u'Connection': u'keep-alive',
              u'Content-Length': u'',
              u'Content-Type': u'',
              u'Host': u'httpbin.org',
              u'User-Agent': u'python-requests/0.14.1 CPython/2.6.8 Darwin/11.4.2'},
 u'origin': u'176.11.12.149',
 u'url': u'http://httpbin.org/get'}

The above example uses http://httpbin.org to reflect the request headers; as you can see requests used a Connection: keep-alive header.