Question

I am using the following function to convert binary data to text:

public string BinaryToText(byte[] data)
{
     MemoryStream stream = new MemoryStream(data);
     StreamReader reader = new StreamReader(stream, encoding.UTF8);
     string text = reader.ReadTod();
     return text;
}

But I get an OutOfMemoryException for 30Mb data.

How can I solve this problem and convert data larger than 100Mb, using this or any better method?

Answer 1

Try this instead:

public string BinaryToText(byte[] data)
{
    return Encoding.UTF8.GetString(data);
}

It will consume less memory. If it still runs out of memory, you’ll need to read it in chunks – but how you are using the data will determine if that is possible. What are you doing with the returned string?

Also I will reiterate my earlier question: Is the input data really UTF8 data?

If you can handle the data being returned as multiple strings, you can do something like this:

public IEnumerable<string> BinaryToStrings(byte[] data, int maxStringLength)
{
    var buffer = new char[maxStringLength];

    using (MemoryStream stream = new MemoryStream(data))
    using (StreamReader reader = new StreamReader(stream, Encoding.UTF8))
    {
        while (true)
        {
            int count = reader.Read(buffer, 0, maxStringLength);

            if (count == 0)
            {
                break;
            }

            yield return new string(buffer, 0, count);
        }
    }
}

Then you can call that in a foreach loop like so:

foreach (string chunk in BinaryToStrings(data, 1024))
{
    // Do something with 'chunk'...
}