Question

I need to get my understanding of character sets and encoding right. Can someone point me to good write up on handling different character sets in C#?

Here’s one of the problems I’m facing –

        using (StreamReader reader = new StreamReader("input.txt"))
        using (StreamWriter writer = new StreamWriter("output.txt")
        {
            while (!reader.EndOfStream)
            {
                writer.WriteLine(reader.ReadLine());
            }
        }

This simple code snippet does not always preserve the encoding –

For example –

Aukéna in the input is turned into Auk�na in the output.

Answer 1

You just have an encoding problem. You have to remember that all you’re really reading is a stream of bits. You have to tell your program how to properly interpret those bits.

To fix your problem, just use the constructors that take an encoding as well, and set it to whatever encoding your text uses.

http://msdn.microsoft.com/en-us/library/ms143456.aspx

http://msdn.microsoft.com/en-us/library/3aadshsx.aspx