Question

I have a crawler application (with C#) that downloads pages from web .
The application take more virtual memory ,
even i dispose every object and even use GC.Collect() .

This , have 10 thread and each thread has a socket that downloads pages .
In each thread , i have a byte[] buffer that store content of page , and have
an string str_content that i store in it , content of page in string .
I have a Sytem.Timer that every 3 second chech , if each thread has been stopped ,
assign it new thread and start it.

I use dispose method and even use GC.Collect() in my application , but in 3 hour my application take
500 MB on virtual memory (500 MB on private bytes in Process explorer) . Then my system
will be hang and i should restart my pc .

1. would it be rude , If i assign my byte[] and string to null ?

2. Is there any way that i use to free virtual memory ?

Thanks .

Answer 1

First of all, you SHOULDN’T be calling gc.collect() anyway, as it is a costly call, and shouldn’t be necessary.

If you are seeing growth AND you are still calling gc.collect() you have resources that still have references, thus they can’t be collected.

I would start looking at your code and making sure that all of your objects are declared at the proper scope, that you are using the Using Statement syntax to ensure that items implementing IDisposable are properly cleaned up and do a full review of your code.

The next step would be to take a tool like ANTS profiler or the like and look at what is actually stored in memory.