Question

I’m using the following code and I can’t figure out why it’s not raising an exception when the urlopen() is failing..

In my particular case, I know why it’s failing.. My url’s don’t have http:// in front of them… but I want to catch those cases so the script can continue to run my code rather than exiting.

    req = urllib2.Request(link)
    try:
        url = urllib2.urlopen(req)
    except urllib2.URLError, e:
        print e.code
        print e.read()
        return False

and I’m getting..

Traceback (most recent call last):
File "./getURLs.py", line 141, in <module>
main()
File "./getURLs.py", line 82, in main
Process(args).get_children()
File "./getURLs.py", line 65, in get_children
self.get_links(link)
File "./getURLs.py", line 46, in get_links
data = urllib2.urlopen(req)
File "/usr/local/lib/python2.7/urllib2.py", line 126, in urlopen
return _opener.open(url, data, timeout)
File "/usr/local/lib/python2.7/urllib2.py", line 383, in open
protocol = req.get_type()
File "/usr/local/lib/python2.7/urllib2.py", line 244, in get_type
raise ValueError, "unknown url type: %s" % self.__original
ValueError: unknown url type: /

.
.
.

Solution

for anyone else interested in my particular solution.. I’m using the following to catch both exceptions.

req = urllib2.Request(link)
    try:
        url = urllib2.urlopen(req)
    except (ValueError,urllib2.URLError) as e:
        print e
        return False

Answer 1

From what you’ve pasted, it looks like you’re catching the wrong type of exception. The code should say

try:
    url=urllib2.urlopen(req)
except ValueError:
    # etc etc etc.

If it’s critical that the entirety of your code run, you can also have a plain except: with an unspecified exception type, or even a finally. See: http://docs.python.org/tutorial/errors.html