We have two identical load balanced web servers (ws1 and ws2) serving the same website. Both have a few crawlers on them at the moment, crawling things like PDFs and pages that contain lots of images (maybe 25/page average). There are ~60 concurrent users between both servers, but the crawlers are by far making the most requests. The IIS App Pool on ws1 is consuming ~500mb RAM and the App Pool on ws2 is consuming ~2gb RAM (4gb available on each server). These consumption levels have been sustained for several hours. I have IIS configured to do static caching and static compression, but not much more I can think of that would impact RAM consumption.
The problem is that ws2’s RAM is just about maxed out and the web pages on that server are loading considerably slower than on ws1, and that’s not good for our end users.
My questions:
- 50% of the available RAM (~2gb on ws2) seems like a lot for an App Pool to consume – is it?
- Are there any programming/configuration considerations I should be making to help the server manage its resources, particularly RAM, better, especially when it comes to larger request like PDFs and images?
I know this is extremely generic at this point. I don’t know what other information would be helpful to include so please leave feedback so I can update this question with better details.
The best starting point is to determine what is using all that memory. Commercial profilers like
http://www.red-gate.com/products/ants_performance_profiler/index.htm
http://www.jetbrains.com/profiler/
can profile your code running in IIS. If you can reproduce the issue in your development environment, the memory profiler of Visual Studio should also be able to diagnose memory usage (perhaps there’s a method to use it stand-alone against a production IIS instance… not sure about that).
Check to see what memory your application has allocated, and for what.