We generate reports in our web application by querying our sql server for data returned as xml and then processing with xslt to create the final output.
As a way to speed up the system, we removed all the static information from the returned sql xml and cached a large XDocument with all the static info in it. Right before performing the xsl transform, we append the XDocument with the static info to the end of the xml that came from sql server. The static XDocument is about 50Meg and takes many seconds to build from the sql server.
Our problem is that once we started caching a few of these large XDoc’s, we hit the cache private bytes limit size and the cache was cleared. Rebuilding these XDocuments is too time consuming to do while people are running reports. I have not tried saving these XDocs to a physical file because they are needed for every report run which happens constantly through the day.
I’ve thought of installing AppFabric Cache but I’m not sure it’s a great idea to store 5 to 10 of these large items in it.
Any ideas? If I install more memory on the web server, will it automatically be available to asp.net for a larger cache? I’ve tried compressing the data before storing it in the cache (shrunk by a factor of 5), but uncompressing it and reparsing the XDocument slowed the server way down.
Final, just save it to a file as it is and then reload it as it is because its all ready Serialized.
The protobuf-net is super fast and light and I have test it and use it, but it not make any good because its all ready serialized.