Question

This is a continuation of the ongoing struggle to reduce my memory load mention in
How do you refill a byte array using SqlDataReader?

So I have a byte array that is a set size, for this example, I’ll say new byte[400000]. Inside of this array, I’ll be placing pdf’s of different sizes (less than 400000).

psuedo code would be:

public void Run()
{
    byte[] fileRetrievedFromDatabase = new byte[400000];
    foreach (var document in documentArray)
    {
        // Refill the file with data from the database
        var currentDocumentSize = PopulateFileWithPDFDataFromDatabase(fileRetrievedFromDatabase);

        var reader = new iTextSharp.text.pdf.PdfReader(fileRetrievedFromDatabase.Take((int)currentDocumentSize ).ToArray());
        pageCount = reader.NumberOfPages;
        // DO ADDITIONAL WORK
    } 
}

private int PopulateFileWithPDFDataFromDatabase(byte[] fileRetrievedFromDatabase)
{
    // DataAccessCode Goes here
    int documentSize = 0;
    int bufferSize = 100;                   // Size of the BLOB buffer.
    byte[] outbyte = new byte[bufferSize];  // The BLOB byte[] buffer to be filled by GetBytes.

    myReader = logoCMD.ExecuteReader(CommandBehavior.SequentialAccess);

    Array.Clear(fileRetrievedFromDatabase, 0, fileRetrievedFromDatabase.Length);

    if (myReader == null)
    {
        return;
    }

    while (myReader.Read())
    {
        documentSize = myReader.GetBytes(0, 0, null, 0, 0);

        // Reset the starting byte for the new BLOB.
        startIndex = 0;

        // Read the bytes into outbyte[] and retain the number of bytes returned.
        retval = myReader.GetBytes(0, startIndex, outbyte, 0, bufferSize);

        // Continue reading and writing while there are bytes beyond the size of the buffer.
        while (retval == bufferSize)
        {
            Array.Copy(outbyte, 0, fileRetrievedFromDatabase, startIndex, retval);

            // Reposition the start index to the end of the last buffer and fill the buffer.
            startIndex += retval;
            retval = myReader.GetBytes(0, startIndex, outbyte, 0, bufferSize);
        }
    }

    return documentSize;
}

The problem with the above code is that that I keep getting a “Rebuild trailer not found. Original Error: PDF startxref not found” error when I try to access the PDF Reader. I believe it’s because the byte array is too long and has trailing 0’s. But since I’m using the byte array so that I’m not continuously building new objects on the LOH, I need to do this.

So how do I get just the piece of the Array that I need and send it to the PDFReader?

Updated

So I looked at the source and realized I had some variables from my actual code that was confusing. I’m basically reusing the fileRetrievedFromDatabase object in each iteration of the loop. Since it’s passed by reference, it gets cleared (set to all zero’s), and then filled in the PopulateFileWithPDFDataFromDatabase. This object is then used to create a new PDF.

If I didn’t do it this way, a new large byte array would be created in every iteration and the Large Object Heap gets full and eventually throws an OutOfMemory exception.

Answer 1

Apparently, I the way the while loop is currently structured, it wasn’t copying the data on it’s last iteration. Needed to add this:

if (outbyte != null && outbyte.Length > 0 && retval > 0)
{
    Array.Copy(outbyte, 0, currentDocument.Data, startIndex, retval);
}

It’s now working, although I will definitely need to refactor.