Question

I’m trying to optimize some code that takes some test data stored in CSV files does some analysis and copies their data into an excel sheet. This code is often run on hundreds of tests at a time, and its taking about 4.5 seconds per test so it can take hours to complete at times.

I looked up some optimization techniques and cut it down by about .25 seconds per test but I think the bulk of the time is being taken up by excel having to “open” the individual files before it can do anything with them. Is there a way to do this more efficiently?

I am open to answers that involve using another language to compile the files into one big file if that would make things quicker.

Answer 1

I would open them as text rather than workbooks:

Sub ReadCSV()
    Dim MyString As String
    Open "C:\path\text.csv" For Input As #1 ' Open file for input. 
    Do While Not EOF(1) ' Loop until end of file.
        Line Input #1, MyString ' Read a line into variable
        Debug.Print MyString ' Print data to the Immediate window.
    Loop
    Close #1 ' Close file.

End Sub

This will be much faster than opening as a workbook