Question

I cannot display theunicode item u'\u201d'. I didn’t have problems with other unicode items. I used UTF-8, but then this character shows up and rained hell on my code. I tried different things in the interpreter. But basically where:

c = u'\u201d'

I get this error:

Traceback (most recent call last):
File "<pyshell#154>", line 1, in <module>
    c.decode('utf-32')
  File "C:\Python27\lib\encodings\utf_32.py", line 11, in decode
    return codecs.utf_32_decode(input, errors, True)
UnicodeEncodeError: 'ascii' codec can't encode character u'\u201d' in position 0: ordinal not in range(128)

I need to display it in the GUI so I can check the output and then store it as plain text.
Transform unicode string in python
explains a bit, however I am still clearly missing something.

Answer 1

If you’re getting this exception, then you’re trying to call .decode() on a unicode string. You should only call .decode() on a byte string, and only call .encode() on a unicode string. Otherwise, the interpreter will first implicitly encode or decode the string using the default codec (usually ‘ascii’), which is bad news.

In general, I recommend reading http://farmdev.com/talks/unicode/ carefully…