Home/ Questions/Q 81585
In Process

The Archive Base Latest Questions

Editorial Team
  • 0
Asked: 2026-05-10T21:28:59+00:00

Basically I need to insert a bunch of data to an Excel file. Creating

  • 0

Basically I need to insert a bunch of data to an Excel file. Creating an OleDB connection appears to be the fastest way but I’ve seen to have run into memory issues. The memory used by the process seems to keep growing as I execute INSERT queries. I’ve narrowed them down to only happen when I output to the Excel file (the memory holds steady without the output to Excel). I close and reopen the connection in between each worksheet, but this doesn’t seem to have an effect on the memory usage (as so does Dispose()). The data is written successfully as I can verify with relatively small data sets. If anyone has insight, it would be appreciated.

initializeADOConn() is called in the constructor

initADOConnInsertComm() creates the insert parameterized insert query

writeRecord() is called whenever a new record is written. New worksheets are created as needed.

public bool initializeADOConn()         {             /* Set up the connection string and connect.*/             string connectionString = @'Provider=Microsoft.Jet.OLEDB.4.0;' +                 'Data Source=' + this.destination + ';Extended Properties=\'Excel 8.0;HDR=YES;\'';             //DbProviderFactory factory =                 //DbProviderFactories.GetFactory('System.Data.OleDb');             conn = new OleDbConnection(connectionString);             conn.ConnectionString = connectionString;             conn.Open();              /* Intialize the insert command. */             initADOConnInsertComm();             return true;         }     public override bool writeRecord(FileListerFileInfo file)             {                 /* If all available sheets are full, make a new one. */                 if (numWritten % EXCEL_MAX_ROWS == 0)                 {                     conn.Close();                     conn.Open();                     createNextSheet();                 }                 /* Count this record as written. */                 numWritten++;                 /* Get all of the properties of the FileListerFileInfo record and add                  * them to the parameters of the insert query. */                 PropertyInfo[] properties = typeof(FileListerFileInfo).GetProperties();                 for (int i = 0; i < insertComm.Parameters.Count; i++)                     insertComm.Parameters[i].Value = properties[i].GetValue(file, null);                 /* Add the record. */                 insertComm.ExecuteNonQuery();                  return true;             }

EDIT:

No, I do not use Excel at all. I’m intentionally avoiding Interop.Excel due to its poor performance (at least from my dabbles with it).

Share

Leave an answer

You must login to add an answer.

Need An Account,

1 Answer

  1. The answer is Yes, the formula you describe does equal a bad time.

    If you have a database handy (SQL Server or Access are good for this), you can do all of your inserts into a database table, and then export the table all at once into an Excel spreadsheet.

    Generally speaking, databases are good at handling lots of inserts, while spreadsheets aren’t.

    • 0