Question

I’m generating a file stream and wrapping it in a buffered stream reader. I am then consuming the stream a line at a time with read line. After X number of lines / bytes I hit the stack overflow exception. It doesn’t seem to be an issue with recursively calling a method as it processes smaller files without issue. I’m hoping I just overlooked something simple here. There’s to much logic to post the entire snippet here but this is the gist…

Instantiates a static stream reader //
{    
    using (FileStream fs = File.Open(filename, FileMode.Open, FileAccess.Read,
    FileShare.Read))
    using (BufferedStream bs = new BufferedStream(fs))
    using (reader = new StreamReader(bs))

     InitializeRecord(reader) // passes reader in
}

InitializeRecord(StreamReader reader)
{  
    //Makes some determinations whether to take in the first line or skip to first    header record... This is working fine. Initializes first line = reader.ReadLine()
    // Calls the first method to generate the header output which in turns calls the LineReader Method to consume the next line.
}

LineReader()
{ // Main loop for iterating over lines where stackoverflow occurs
    while (!reader.EndOfStream)
    {
        string prev_line = line;
        line = reader.ReadLine(); // StackOverFlow occurs here only on larger files / # of bytes read        
        VerifyLine(line,prev_line);
    }
}

VerifyLine(string line)
{ 
    // Does some checking on the line and calls output methods for each record type which in turn calls LineReader which LineReader exits when the endofstream is reached. 
    //But is blowing up prior to reaching the end of the stream.  By writing the lines out to disk as it iterates it writes a replica of the stream perfectly until the stack overflow occurs. 
    //This is only the difference of anything greater than a 5 MB file. Some of these records are hitting 9 million characters. I tried increasing the buffer size without luck.
}

Answer 1

It doesn’t seem to be an issue with recursively calling a method as it
processes smaller files without issue

But you are saying it blows up on larger files correct? For me, this sounds like there is a problem with your recursion. Is there anyway to perform your operations without recursion? I’d like to see more code in the verifyLine method.