Question

Why am I getting this issue? and how do I resolve it?

UnicodeDecodeError: 'utf8' codec can't decode byte 0x92 in position 24: unexpected code byte

Thank you

Answer 1

Somewhere, perhaps subtly, you are asking Python to turn a stream of bytes into a “string” of characters.

Don’t think of a string as “bytes”. A string is a list of numbers, each number having an agreed meaning in Unicode. (#65 = Latin Capital A. #19968 = Chinese Character “One”/”First”) .

There are many methods of encoding a list of Unicode entities into a stream of bytes. Python is assuming your stream of bytes is the result of a particular such method, called “UTF-8”.

However, your stream of bytes has data that does not correspond to that method. Thus the error is raised.

You need to figure out the encoding of the stream of bytes, and tell Python that encoding.

It’s important to know if you’re using Python 2 or 3, and the code leading up to this exception to see where your bytes came from and what the appropriate way to deal with them is.

If it’s from reading a file, you can explicity deal with the bytes read. But you must be sure of the file encoding.

If it’s from a string that is part of your source code, then Python is assuming the “wrong thing” about your source files… perhaps $LC_ALL or $LANG needs to be set. This is a good time to firmly understand the concept of encoding, and how text editors choose an encoding to write, and what is standard for your language and operating system.