Question

I am working on the exel parsing using python.
till now I have worked with english language but when I encounter the regional languages, I am getting the error.

example :

IR05 měsíční (monthly)

It gives me the error as

UnicodeEncodeError: 'ascii' codec can't encode character u'\u011b' in position 6: ordinal not in range(128)

how I can parse it and I can again write in same language in output files?

my code :

for j in val:
    print 'j is - ', j
    str(j).replace("'", "")

I am getting error at replace statement.

Answer 1

>>> "IR05 měsíční (monthly)".decode('utf8')
u'IR05 m\u011bs\xed\u010dn\xed (monthly)'

which is a unicode version of your original string (which was encoded in utf8).

Now you can compare it to your other string (from the file), which you decode (from utf8 or latin2 or a different format) and you can compare them.

>>> 'IR05 m\xecs\xed\xe8n\xed (monthly)'.decode('latin2')
u'IR05 m\u011bs\xed\u010dn\xed (monthly)'

now you can compare the two unicode strings:

>>> s_utf8 = "IR05 měsíční (monthly)"
>>> s_latin2 = 'IR05 m\xecs\xed\xe8n\xed (monthly)'
>>> s_utf8.decode('utf8') == s_latin2.decode('latin2')
True

To write the string into a file, encode it again:

>>> s = s_utf8.decode('utf8')

>>> filehandle.write(s.encode('utf8'))