Question

I am experiencing a bad memory leak which occurs in the following code:

 public void BulkInsert(string tableName, IDataReader reader, String connectionString)
    {
        using (var connection = new SqlConnection(connectionString))
        {
            connection.Open();
            using (var bulkCopy = new SqlBulkCopy(connection))
            {
                bulkCopy.DestinationTableName = tableName;
                bulkCopy.BulkCopyTimeout = 900;

                try
                {
                    bulkCopy.WriteToServer(reader);
                }
                catch (Exception ex)
                {
                    Console.WriteLine(ex.Message);
                }
                finally
                {
                    reader.Close();
                }
            }
        }
    }

This code segment is executed thousands of times and as such it only takes a minute for an out of memory exception to occur. Ants reports that this is because the rows pointed to by the IDataReader are not being collected by the GC. However when I comment out the following line there is no leak which is how i’ve isolated the issue to this peice of code.

bulkCopy.WriteToServer(reader);

Has anyone a suggestion as to how to prevent the memory leak?

Thank you in advance.

Calling Code:

 var reader = datatable.CreateDataReader();
 BulkInsert(tablename, reader, connectionString);
 reader.Dispose();
 datatable.Dispose();

Answer 1

I absolutely hate answering my own question but I finally found the solution and although I feel dumb for making this mistake but I just wanted to post it as the answer in case anyone else comes across this question and so nobody else wastes valuable time on this.

The Memory Leak was not with the provided code segment at all – In fact it was not a leak at all…

Looking at the Performance Analysis I discovered that speedwise the BulkCopy call was the bottleneck of my whole program. I have a Producer-Consumer pattern feeding it DataTables for it to insert.

I thought the Memory Profiler was showing me DataTable objects which were not being disposed of when I ran the code. These were in-fact queued tables waiting to be inserted but because I was re-using test data the queued tables were already in the DB (and as such appeared to have already been inserted).

By commenting out the BulkCopy line I was in effect removing the bottleneck, the DataTables were being quickly disposed, and thus no issues showed on the Memory Profiler. This made it appear like the Bulkcopy line was at fault.

When I replaced the BulkCopy code with a 1s delay this was not enough of a bottleneck. It was only when I replaced the BulkCopy with a 5s delay that I noticed my Consumer Producer Queue Size getting out of control…. Hence why I was quickly running out of memory.

Thanks to all who took time on this question. I’m sorry it did not have a more interesting answer.