Question

My code fetches CSV data from a PHP page using httplib. When I open the page in Firefox or Chrome, the data displays just fine. However, when I try to fetch it with my python code, I get a header with content-length: 0 and no data. This page is the only one that does this – in another page in the same directory, the python httplib fetching works just fine. Can someone tell me what I’m doing wrong?

code:

FILE_LOC = '/core/csv.php'
argstr = '?type=' + self.type + '&id=' + self.id
conn = httplib.HTTPConnection(SERVER_ADDRESS)
conn.request('GET', FILE_LOC + argstr)
resp = conn.getresponse()
csvstr = resp.read()

The response headers:

[('content-length', '0'), ('x-powered-by', 'PHP/5.1.6'),
('server', 'Apache/2.2.3 (CentOS)'), ('connection', 'close'),
('date', 'Thu, 19 Aug 2010 21:39:44 GMT'), ('content-type', 'text/html; charset=UTF-8')]

Answer 1

Perhaps the PHP script expects to see some HTTP header or headers that the httplib module isn’t sending. For example, httplib does not seem to send Accept, Accept-Language, or User-Agent headers by default. You may need to add one or more of those to the request() call. It does seem to send a proper Host header, though, which was my first guess.