Question

I am using the python package requests to make a get request to a server. My goal is to replicate certain known http requests and incorporated them into my python script. The original query string contains one key-value pair ‘version=0%2E12’ which I know is the url encoded equivalent of ‘version=0.12’. Below is an example of what I am doing to replicate the http request.

params = {
          'version'='0.12',
          'language'='en',
          ...
          }
 resp = requests.get(url, params)

The resulting url query string has ‘version=0.12’ in it. If I try to change the setting of params to the following,

    params = {
          'version'='0%2E12',
          'language'='en',
          ...
          }

but the resulting query string contains ‘version=0%252E31’. Essentially, requests is parsing the % to be url encoded.

How can I get requests to parse my params arguments with periods properly without manually writing the entire url?

Answer 1

Note that . is a perfectly valid character in a URL-encoded value.

You’d need to encode the values yourself, bypassing the standard encoding. I’d use urllib.urlencode(), then replace the . characters by %2E by hand.

You then have to append the encoded string as a query parameter to the URL, and configure requests to not re-encode the values:

params = {
      'version'='0.12',
      'language'='en',
      ...
      }
params = urllib.urlencode(params)
params = params.replace('.', '%2E')
resp = requests.get('?'.join(url, params), config={'encode_uri': False})

The encode_uri parameter set to False does the trick here; it’s this requests default that otherwise will re-encode the whole URL to ensure RFC compliancy.