Question

I have a byte array as input. This should be the binary representation of standard conforming UTF8 HTML. It is, but most of the time only. Sometimes it also contains embedded nulls (\x0 character or NUL). This is not under my control. I need to transform this byte array into a string.

Tried so far:

• Obviously using a StreamReader or TextReader does not work as it stops on hitting the first NUL
• Encoding.UTF8.GetString does not work either – also stops on the first NUL

What worked but is rather unelegant:

   mynewarray = myoldarray.Where( x => x!=0).ToArray();
   var output = Encoding.UTF8.GetString(mynewarray);

Is there a more elegant way to do this excepted in creating a new byte array skipping the NUL chars and then use one of the solutions above? The byte array can be pretty big, more than 2-4 Mb… The MSDN tells that Strings might actually contain embedded NUL but does not tell what are the best approaches to handle such strings.

Answer 1

Your string is already right. It will contain the NUL characters. But when you use a string with the included NUL chars you will get all kind of problems.

Encoding.UTF8.GetString does not stop at \0 as you see in my example.

See what happens when I output such a string:

  var text = new byte[]{65, 65, 0, 65};
  var s = Encoding.UTF8.GetString(text);
  Console.WriteLine("len is: " + s.Length + " chars");
  Console.WriteLine("text: '" + s + "'");      
  Console.WriteLine("this line doesn't appear because NUL was sent to console");

output is:

len is: 4 chars
text: 'AA