Home/ Questions/Q 7539821
In Process

The Archive Base Latest Questions

Editorial Team
  • 0
Asked: 2026-05-30T07:29:03+00:00

I’m working with CSV transaction data files that are 350 mb+ and 1,100,000+ lines

  • 0

I’m working with CSV transaction data files that are 350 mb+ and 1,100,000+ lines each.

I was wondering how can I perform some simple fast queries on these files through the VBA IDE, save the CSV, and then open the result as a workbook in Excel.

For example, I want to do this:

  1. Load the CSV into RAM as a table
  2. Remove all rows where the field called transaction_type is recorded as “failed”
  3. Save the result as a new CSV
  4. Open the result as a workbook in Excel

My goal is to do this operation with the highest performance possible. I think that this functionality is provided by the Extensible Storage Engine (ESE), but I’m not sure how to use it through the Excel VBA IDE.

Thanks!

Share

Leave an answer

You must login to add an answer.

Need An Account,

1 Answer

  1. Editorial Team

    You could use a ‘text database’ and use ADO (or DAO) to query the files. See this article for more information: http://msdn.microsoft.com/en-us/library/ms974559.aspx

    That way you can just create a schema.ini file for the file you wish to query and query the file using standard SQL. You would then simply write your result recordset to file.

    • 0