We’re building a text templating engine out of a custom HttpModule that replaces tags in our HTML with whole sections of text from an XML file.
Currently the XML file is loaded into memory as a string/string Dictionary so that the lookup/replace done by the HttpModule can be performed very quickly using a regex.
We’re looking to expand the use of this though to incorperate larger and larger sections of replaced text and I’m concerned over keeping more verbose text in memory at one time as part of the Dictionary, especially as we use ASP.NET caching for many uses as well.
Does anyone have a suggestion for a more efficient and scalable data structure/management strategy that we could use?
UPDATE: In response to Corbin March’s great suggestion below, I don’t think this is a case of us going down a ‘dark road’ (although I appreciate the concern). Our application is designed to be reskinned completely for different clients, right down to text anywhere on the page – including the ability to have multiple languages. The technique I’ve described has proven to be the most flexible way to handle this.
The amount of memory you are using is going to be roughly the same as the size of the file. The XML will have some overhead in the tags that the Dictionary will not, so it’s a safe estimate of the memory requirements. So are you talking about 10-50 MB or 100-500 MB? I wouldn’t necessarily worry about 10 to 50 MB.
If you are concerned, then you need to think about if you really need to do the replacements everytime the page is loaded. Can you take the hit of going to a database or the XML file once per page and then caching the output of the ASP.NET page and hold it for an hour? If so, consider using Page Caching.