Question

I use a 3rd party tool that outputs a file in Unicode format. However, I prefer it to be in ASCII. The tool does not have settings to change the file format.

What is the best way to convert the entire file format using Python?

Answer 1

You can convert the file easily enough just using the unicode function, but you’ll run into problems with Unicode characters without a straight ASCII equivalent.

This blog recommends the unicodedata module, which seems to take care of roughly converting characters without direct corresponding ASCII values, e.g.

>>> title = u'Klüft skräms inför på fédéral électoral große' 

is typically converted to

Klft skrms infr p fdral lectoral groe 

which is pretty wrong. However, using the unicodedata module, the result can be much closer to the original text:

>>> import unicodedata >>> unicodedata.normalize('NFKD', title).encode('ascii','ignore') 'Kluft skrams infor pa federal electoral groe'