Question

I am trying to build a url so that I can send a get request to it using urllib module.

Let’s suppose my final_url should be

url = "www.example.com/find.php?data=http%3A%2F%2Fwww.stackoverflow.com&search=Generate+value"

Now to achieve this I tried the following way:

>>> initial_url = "http://www.stackoverflow.com"
>>> search = "Generate+value"
>>> params = {"data":initial_url,"search":search}
>>> query_string = urllib.urlencode(params)
>>> query_string
'search=Generate%2Bvalue&data=http%3A%2F%2Fwww.stackoverflow.com'

Now if you compare my query_string with the format of final_url you can observer two things

1) The order of params are reversed instead of data=()&search= it is search=()&data=

2) urlencode also encoded the + in Generate+value

I believe the first change is due to the random behaviour of dictionary. So, I though of using OrderedDict to reverse the dictionary. As, I am using python 2.6.5 I did

pip install ordereddict

But I am not able to use it in my code when I try

>>> od = OrderedDict((('a', 'first'), ('b', 'second')))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'OrderedDict' is not defined

So, my question is what is the correct way to use OrderedDict in python 2.6.5 and how do I make urlencode ignores the + in Generate+value.

Also, is this the correct approach to build URL.

Answer 1

You shouldn’t worry about encoding the + it should be restored on the server after unescaping the url. The order of named parameters shouldn’t matter either.

Considering OrderedDict, it is not Python’s built in. You should import it from collections:

from urllib import urlencode, quote
# from urllib.parse import urlencode # python3
from collections import OrderedDict

initial_url = "http://www.stackoverflow.com"
search = "Generate+value"
query_string = urlencode(OrderedDict(data=initial_url,search=search))
url = 'www.example.com/find.php?' + query_string 

if your python is too old and does not have OrderedDict in the module collections, use:

encoded = "&".join( "%s=%s" % (key, quote(parameters[key], safe="+")) 
    for key in ordered(parameters.keys()))

Anyway, the order of parameters should not matter.

Note the safe parameter of quote. It prevents + to be escaped, but it means , server will interpret Generate+value as Generate value. You can manually escape + by writing %2Band marking % as safe char: