Question

I am grabbing a value at a time and dynamically loading it into a grid.

Is there a way to index a csv file to only look up a value at a certain row and column?
I can’t read all the rows as that would defeat the purpose of loading dynamically.

The CSV parser, Fast CSV Parser in my case, can grab a value like so csv[row][column]. When looking at the source, I noticed that it loops over everything in the file until it reaches the correct index column pair. To grab a value at row 100,000 column 80, it can take quite a long time.

Any help much appreciated.

Answer 1

Well, you could do a fast first pass and store the offsets of each row. That would make subsequently locating a row much faster. If you have 80 columns but 100K rows, I’d focus on fast row-finding rather than fast column-finding.

ETA: OK, I’m assuming your CSV file is on disk and that you can get exclusive access to it. Some of this code was based on this.

   List<int> offsets = new List<int>();
   using (StreamReader reader = new StreamReader("myfile.csv"))
   {
        int offset = 0;
        string line;
        while ((line = reader.ReadLine()) != null)
        {   
            offsets.Add(offset);             
            offset += (line.Length + 2);   // The 2 is for NewLine(\r\n)
        }
        offsets.Add(offset);  // pick up the last one
    }

At the end of this, you will have the List variable offsets, which is indexed by line number and contains the offset to each line. You can then, when reading the file (when doing your grid-building) use offsets[n] to get the offset to Seek to (I’m assuming you’re using a FileStream or a StreamReader) and offsets[n+1] - offsets[n] to get the length.

As far as parsing the returned line of text is concerned, I assume the CSV library you’re adapting has good logic for that.