Question

I want to catch a urllib2.HTTPError with extra information if it’s a 404:

try:
    data = urlopen(url)
except HTTPError, e:  # Python 2.5 syntax
    if e.code == 404:
        raise HTTPError('data not found on remote')
    else:
        raise

but this doesn’t work because HTTPError‘s init takes multiple arguments, which are undocumented. It it did work, it would lose the backtrace and the original message. I also tried

if e.code == 404:
    e.message = 'data not found on remote: %s' % e.message
raise

but that just re-raised the exception without extra information. What should I do?

Answer 1

You just need to use e.msg rather than e.message. The script:

from urllib2 import urlopen, HTTPError

url = 'http://www.red-dove.com/frob'

try:
    data = urlopen(url)
except HTTPError, e:  # Python 2.5 syntax
    if e.code == 404:
        e.msg = 'data not found on remote: %s' % e.msg
    raise

prints

Traceback (most recent call last):
  File "c:\temp\test404.py", line 6, in <module>
    data = urlopen(url)
  File "C:\Python\Lib\urllib2.py", line 124, in urlopen
    return _opener.open(url, data)
  File "C:\Python\Lib\urllib2.py", line 387, in open
    response = meth(req, response)
  File "C:\Python\Lib\urllib2.py", line 498, in http_response
    'http', request, response, code, msg, hdrs)
  File "C:\Python\Lib\urllib2.py", line 425, in error
    return self._call_chain(*args)
  File "C:\Python\Lib\urllib2.py", line 360, in _call_chain
    result = func(*args)
  File "C:\Python\Lib\urllib2.py", line 506, in http_error_default
    raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
urllib2.HTTPError: HTTP Error 404: data not found on remote: Not Found

You can of course tidy this up with an enclosing try/except:

from urllib2 import urlopen, HTTPError

url = 'http://www.red-dove.com/frob'

try:
    try:
        data = urlopen(url)
    except HTTPError, e:  # Python 2.5 syntax
        if e.code == 404:
            e.msg = 'data not found on remote: %s' % e.msg
        raise
except HTTPError, e:
    print e

which prints simply

HTTP Error 404: data not found on remote: Not Found

The exception has all of the original detail: e.__dict__ looks like

{'__iter__': <bound method _fileobject.__iter__ of <socket._fileobject object at   0x00AF2EF0>>,
 'code': 404,
 'fileno': <bound method _fileobject.fileno of <socket._fileobject object at 0x00AF2EF0>>,
 'fp': <addinfourl at 12003088 whose fp = <socket._fileobject object at 0x00AF2EF0>>,
 'hdrs': <httplib.HTTPMessage instance at 0x00B727B0>,
 'headers': <httplib.HTTPMessage instance at 0x00B727B0>,
 'msg': 'data not found on remote: Not Found',
 'next': <bound method _fileobject.next of <socket._fileobject object at 0x00AF2EF0>>,
 'read': <bound method _fileobject.read of <socket._fileobject object at 0x00AF2EF0>>,
 'readline': <bound method _fileobject.readline of <socket._fileobject object at 0x00AF2EF0>>,
 'readlines': <bound method _fileobject.readlines of <socket._fileobject object at 0x00AF2EF0>>,
 'url': 'http://www.red-dove.com/frob'}