Question

I’ve got a system that processes some large CSV files.

The scenario has now arisen where these files might have a number of none-delimited, worthless lines preceding the actual comma-delimited content.

The approach I have taken is to create a temporary reader to ascertain the number of superfluous lines and then move the working TextReader on that number of lines ready to be processed.

My code is as follows:

private static TextReader PrepareReader(TextReader reader)
    {
        // Variables
        TextReader tmpReader = reader;
        Int32 superfluousLineCount = 0;

        // Determine how many useless lines we have
        using (tmpReader)
        {
            string line;
            string headerIdentifier = "&1,";
            while ((line = tmpReader.ReadLine()) != null)
            {
                // Check if the line starts with the header row identifier
                if (line.Substring(0, 3) != headerIdentifier)
                {
                    // Increment the superfluous line counter
                    superfluousLineCount++;
                }
                else
                {
                    break;
                }
            }
        }

        // Move the source reader through how many lines we want to ignore
        using (reader)
        {
            for (int i = superfluousLineCount; i > 0; i--)
            {
                reader.ReadLine();
            }
        }

        // Return
        return reader;
    }

However, the reader.ReadLine(); in this part of the code:

for (int i = superfluousLineCount; i > 0; i--)
{
reader.ReadLine();
}

…throws the following exception

Cannot read from a closed TextReader.
ObjectDisposedException in mscorlib
Method:
Void ReaderClosed()

Stack Trace:
at System.IO.__Error.ReaderClosed()
at System.IO.StreamReader.ReadLine()
at CsvReader.PrepareReader(TextReader reader) in CsvReader.cs:line 93

Any advice greatly appreciated. Also, is the best way to go about my challenge?

Notes: Framework 2.0

Thanks.

Answer 1

When you are using using (tmpReader) it will close tmpReader (Which references the same object as reader does), so when you try to read from reader in your loop, it is closed.

Your best bet is to combine the two loops. Sine you only want to skip lines, I would think the logic of the first loop is sufficient.