Question

These have been plaguing me endlessly. Why? It seems that my console can’t handle the encoding. I take it that the my browser and word processor can handle it. I don’t have a master list of all the possible characters that it’s choking on. What is the best way to relieve this without modifying my data?

'charmap' codec can't encode character u'\xca'

Answer 1

You need to find out the encoding of your console (which system, OS, etc…?) — 'charmap' is unfortunately a somewhat-ambiguous identification for a codec, as the docs explain:

There’s another group of encodings
(the so called charmap encodings) that
choose a different subset of all
unicode code points and how these
codepoints are mapped to the bytes
0x0-0xff. To see how this is done
simply open e.g. encodings/cp1252.py
(which is an encoding that is used
primarily on Windows). There’s a
string constant with 256 characters
that shows you which character is
mapped to which byte value.

All of these encodings can only encode
256 of the 65536 (or 1114111)
codepoints defined in unicode.

i.e., it identifies a set of possible codecs, not a specific one.

Once you know your console supports a codec named 'foobar', change your statements that are now

print(someunicode)

into

print(someunicode.encode('foobar'))