I’ve written a small, in house C# program which batch converts between various file formats, mostly relying on other API’s. Currently, the UI spawns a BackgroundWorker (not a Thread) to handle the conversions, and then fills a queue with the requests which empties as the worker completes jobs. The queue object themselves are very small (3 Strings to tell the worker what to do) and don’t contribute greatly to the program’s memory footprint. I’ve been fairly draconian with my memory management (disposing of Images when finished with them, and manually garbage collecting at certain points.) Still, the program tends to use about 100 MB of memory at a time, and uses about 50% of total CPU time. It seems like if I naively implemented threading, it would quickly run out of system memory (unless the CLR does some sort of magic I don’t know about.)
Is there a simple/efficient way to spawn threads to prevent the system from running out of memory besides catching OutOfMemory exceptions and rolling back the thread that died (seems very inefficient, but there’s no way to preserve state without using prohibitive amounts of memory)?
If you use the ThreadPool.QueueUserWorkItem to spawn the conversions you will automatically get a limit on how many threads that are running. The ThreadPool manages this internally and will process queued calls as soon as a pool thread becomes available.