Question

I’m using the python module requests to get data from some API’s and they all return json data which are converted to dicts. What I want to do is take some info from these dicts and either convert them all to python strings where I can use the stemming and string.translate() modules on them, or convert the whole thing to data that is recognisable to these modules. I can’t do this with the UTF-8 data and it’s doing my head in. Is there any solution to this at all? Can I iterate through the dict and convert it to ASCII?

The strange thing is I am comparing ASCII strings to the UTF data in other functions (if ASCII-word is in UTF dict: do something) and it works perfectly. The ASCII value matches the UTF-8 data all the time. I can’t get my head around this encoding stuff at all

Answer 1

When you are sure the function does not support Unicode, you can always convert to an ASCII-approximation:

ascii_string = unicodedata.normalize('NFKD', unicode_string).encode('ascii','ignore')