Question

I’m looking for a quick way to get an http response code from a url. If code is 200′ then download the images. Can i get response code with MyOpener`? tahnks

from urllib import FancyURLopener
class MyOpener(FancyURLopener):
  version = 'Mozilla/5.0 (Windows; U; Windows NT 5.1; it; rv:1.8.1.11) Gecko/20071127 Firefox/2.0.0.11'


myopener = MyOpener()
myopener.retrieve('http://upload.wikimedia.org/wikipedia/en/4/44/Zindagi1976.jpg', 'Zindagi1976.jpg')

UPDATE:

>>> import urllib
>>> resp = urllib.urlopen("http://upload.wikimedia.org/wikipedia/en/4/44/Zindagi1976.jpg")
>>> print resp.getcode()
 403

Answer 1

What’s wrong with this or I got your question wrong.

>>> import urllib
>>> resp = urllib.urlopen("http://docs.python.org/library/urllib.html")
>>> if resp.getcode() == 200:
...     print "do my stuff"
...
do my stuff
>>>

It’s nice that you have worked your way around the problem. There is a reason that wikimedia gives 403 as a response code. The reason is as soon as you send a request to access the content of wikimedia it realizes that this request is not send by a browser so it throws a 403 error.

Websites do this type of checking to make sure contents are not being accessed by bots. There are many other checks and User-Agent is one of them.

So, to make it like a browser is sending a request you can add User-Agent to your python code.

>>> import urllib2
>>> req = urllib2.Request('http://upload.wikimedia.org/wikipedia/en/4/44/Zindagi1976.jpg')
>>> useragent = 'Mozilla/5.0 (Windows; U; Windows NT 5.1; it; rv:1.8.1.11) Gecko/20071127 Firefox/2.0.0.11'
>>> req.add_header('User-Agent',useragent)
>>> resp = urllib2.urlopen(req)
>>> resp.getcode()
200
>>> data  = resp.read()
>>> with open("image.jpg","wb") as f:
...     f.write(data)
...
>>>