Question

I am reading binary data from db and converting it into text by using code.

       public String BinaryToText(byte[] data)
       {
         System.Text.Encoding encEncoder = System.Text.ASCIIEncoding.ASCII;

         return encEncoder.GetString(data);
       }

The above procedure is correctly working but when binary file >= 85mb is converted , OutOfMemoryException
is shown.How to convert large binary data into string without error.

Answer 1

I wouldn’t normally expect 85MB too be a problem unless you are low on memory etc. Even x86 can usually handle that without pausing for breath.

For large amounts of data, the simplest answer is always “don’t hold it all in memory at once”. ADO.NET has a forwards-only API on data-reader that allows successive calls to fetch different parts of a large BLOB:

using(var reader = cmd.ExecuteReader(CommandBehavior.SequentialAccess))
{  //                                    ^^ forwards-only mode ^^
  long offset = 0;
  int read;
  byte[] buffer = new byte[8096];
  while ((read = reader.GetBytes(colIndex, offset, buffer, 0, buffer.Length))>0)
  {
     ProcessBytes(buffer, 0, read);
     offset += read;
  }
}

where ProcessBytes(byte[] buffer, int offset, int count) processes count bytes from buffer, starting at offset. In the case of ASCII, you can probably get away without using an encoding at all; for other encodings, you can use the Encoding.GetDecoder() API to decode a stream of data, although it is a bit messy. These two things together will allow you to handle an arbitrarily large (multi-terabyte if necessary) data-source without having it all in memory at once.

The next question, then, is: what are you going to do with this data?

If you do need it all in memory at once, you have no choice but to hold it. You might be able to do something with an iterator block, to return fragments of the string in turn.