Question

I have been using PHP for a while,

but now I am trying to learn Python.
I never really had a serious problem with encoding on PHP. At least I could solve in few minutes. but I have spent hours to solve encoding problems in python, and I still can’t solve it.

So my problem is that I want to print a Japanese string to a file using Shell redirect using this command

python make_csv.py > csvfile.csv

But the Japanese texts that are printed on the csv file are all messed up.

I’ve tried several solutions like using .encode(‘utf-8’) etc, but still no luck
and I also encounter this error:

UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 5: ordinal not in range(128)

how can I solve this?
please help.

Answer 1

Try setting your environment variable PYTHONIOENCODING to utf8. That will make Python use that encoding instead of none by default.