Question

when i am working with urllib2 module not in function everything work good (here is code):

headers = {'User-Agent' : 'Mozilla/5.0 (Windows NT 6.1) AppleWebKit/535.7 (KHTML, like Gecko) Chrome/16.0.912.77 Safari/535.7'}
request = urllib2.Request(url, '', headers)
response = urllib2.urlopen(request)
returned_array = [response.geturl(), response.read()]
print returned_array

but when i put this to function:

def nk_get_site(url):
  headers = {'User-Agent' : 'Mozilla/5.0 (Windows NT 6.1) AppleWebKit/535.7 (KHTML, like Gecko) Chrome/16.0.912.77 Safari/535.7'}
  request = urllib2.Request(url, '', headers)
  response = urllib2.urlopen(request)
  returned_array = [response.geturl(), response.read()]
  return returned_array

it dont work, error:

Traceback (most recent call last):
File "C:\py\NetKit\netkit.py", line 37, in <module>
print nk_get_site('http://www.google.com/')
File "C:\py\NetKit\netkit.py", line 33, in nk_get_site
response = urllib2.urlopen(request)
File "C:\Python27\lib\urllib2.py", line 126, in urlopen
return _opener.open(url, data, timeout)
File "C:\Python27\lib\urllib2.py", line 400, in open
response = meth(req, response)
File "C:\Python27\lib\urllib2.py", line 513, in http_response
'http', request, response, code, msg, hdrs)
File "C:\Python27\lib\urllib2.py", line 438, in error
return self._call_chain(*args)
File "C:\Python27\lib\urllib2.py", line 372, in _call_chain
result = func(*args)
File "C:\Python27\lib\urllib2.py", line 521, in http_error_default
raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
HTTPError: HTTP Error 405: Method Not Allowed

Answer 1

This has nothing to do with you putting the code in a function, and everything to do with the URL you’re testing:

>>> headers = {'User-Agent' : 'Mozilla/5.0 (Windows NT 6.1) AppleWebKit/535.7 (KHTML, like Gecko) Chrome/16.0.912.77 Safari/535.7'}
>>> request = urllib2.Request(url, '', headers)
>>> response = urllib2.urlopen(request)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Python25\lib\urllib2.py", line 124, in urlopen
    return _opener.open(url, data)
  File "C:\Python25\lib\urllib2.py", line 387, in open
    response = meth(req, response)
  File "C:\Python25\lib\urllib2.py", line 498, in http_response
    'http', request, response, code, msg, hdrs)
  File "C:\Python25\lib\urllib2.py", line 425, in error
    return self._call_chain(*args)
  File "C:\Python25\lib\urllib2.py", line 360, in _call_chain
    result = func(*args)
  File "C:\Python25\lib\urllib2.py", line 506, in http_error_default
    raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
urllib2.HTTPError: HTTP Error 405: Method Not Allowed

By passing '' to the data parameter, you’re turning it into a POST request. If you just want to pass headers on a GET request, you should be using this:

request = urllib2.Request(url, headers=headers)