Question

I have a web application that creates XML feeds on the fly, depending on criteria passed in via the query string.

It is a very simple application: it reads the query string variables, and uses them to format data (cached data from the database) into the appropriate XML format.

The returned XML file is about 25MB… there is a lot of data.

I am using an XmlTextWriter to build the XML, and then returning that to the requestor as an “application/octet-stream” – an attachment they can download.

Question is: Building the XML seems to use 100% of the CPU and is causing problems with my other applications.

Does anyone have experience with building such large XML files? Any tips on making the process less CPU intensive?

Code sample is:

    map.WriteRaw("<?xml version=\"1.0\" encoding=\"UTF-8\"?>");
            map.WriteStartElement("rss");
            map.WriteAttributeString("version", "2.0");
            map.WriteAttributeString("xmlns:g", "http://base.google.com/ns/1.0");
            map.WriteStartElement("channel");
            map.WriteElementString("link", "http://www.mywebsite.com...");

            ProductCollection items = Product.GetCachedSiteProducts();
            foreach (Product p in items)
            {
map.WriteStartElement("item");
........
                    map.WriteElementString("description", p.Description);
                    map.WriteElementString("g:id", p.SiteSku);
                    map.WriteElementString("g:condition", "new");
                    map.WriteElementString("g:price", p.Price.ToString() + " USD");
...............

                    map.WriteEndElement(); //item            
                }
            }
            map.WriteEndElement();//channel    
            map.WriteEndElement();//rss    
            Response.Write(sw.ToString());

UPDATE: I am answering my own question.. thanks to those who asked me to post more code, this was an easy one when I looked more carefully.

Code was using “Response.write(map.ToString())” to output the xml. Wow, that’s inefficient. Will update the code. Thanks all!

Answer 1

One immediate concern that springs to mind is Response.Writer(sw.ToString())

This looks like you are writing to a StringWriter first then sending to the output stream, why not write directly to the output stream here?

I wouldn’t expect that alone to make much difference to CPU usage or cause 100% CPU usage.

What is ProductCollection, as you loop over that looks to be the most likely cause of the high CPU usage. If your code is properly doing IEnumerable<Product> and obtaining products JIT this may cause a lot of CPU usage if you are obtaining the products for some persistent storage (e.g. each iteration requires some independent database calls)

Without seeing further source code it’s hard to tell what might be the cause of the problem.