Question

I keep getting System.OutOfMemory exception in my WinForms App which is thrown while doing an SQL Update I guess. How can I fix that? Here is the case, I am using SQLEXPRESS, trying the code on a 32-bit machine so ThreadPool has default 1023 avaliable worker threads on startup.

MyClass myClass = new MyClass();

for (int i = 0; i < 1000000; i++)
{              
    ThreadPool.QueueUserWorkItem(new WaitCallback(Foo), myClass);
}

private void Foo(object state)
{
    //Some Stuff and SQL UPDATE
}

When the application starts, the ThreadPool starts with 8 threads and starts to increase allocated thread number to perform the job. A while later, it comes to 200 threads for example and can handle it no more, gives out System.OutOfMemory exception. When I check the stack trace, i can see that exception occurs in SQL operation of the method. What should I do? Do I need to increase Buffer Size of my database? I do not want to limit ThreadPools max size by the way or try Thread.Sleep() to have slower, less frequent requests to DB.

Answer 1

Blindly creating lots of threads to make an operation go faster is not a good solution and almost never works (except by luck maybe). Keep in mind that each threads gets its own stack allocated which is 1MB by default I think. So in case you get 1000 threads going that will use 1GB of RAM just like that.

If you hit the DB then the work you can do in parallel will be very likely limited by disk I/O so throwing more threads at it might actually make it worse.

Also be aware that asynchroneous operations are executed on threadpool threads and if you tie them all up with your own work you may run into the problem to starve those operations (meaning they might be never or very late executed). The threadpool is designed to run only short lived tasks. If you need long running tasks then use a different threadpool (for example the SmartThreadPool) or create you own set of threads to handle the work.

Depending on what your SQL operations are you might hit the Large Object Heap fragmentation problem. Objects larger than 85.000 bytes are put on the LOH which is not being compacted and you can run into unexpected OOM ecceptions. So check if you are creating large arrays or lists of objects.

Otherwise: Use the debugging tools for windows to make a memory dump and look at what objects you have settings around eating all the memory and what holds the references to keep them alive. Alternatively you could use a .NET memory profiler but most of the really usefull ones are not free (however they usually come with a X days evaluation period).