Question

I have an xml file with 3 pieces of data per item: productName, productCode and productPrice. The number of items in the xml file is around 7,000.

User can search for product by name by typing “red shirt” for example in a textbox.
My code does these tasks:

1. tokenize the search text and collect productName’s that contains ALL of the search words.

2. order the selection by productCode

3. select for display only productName and productPrice (not productCode which is used only for orderby)

  var products = from d in xElem.Descendants(fileName)
                        where textBox1.Text.ToLower().Split(' ').All(t => d.Element(productName).Value.ToLower().Contains(t))

                        orderby d.Element(productCode).Value ascending
                        select new
                        {
                            Price = (double.Parse(d.Element(productPrice).Value)).ToString(numberFormat),
                            Name = d.Element(Name).Value
                        };

Where in this code are the bottleneck(s)? And how to remove them?
I do this Linq select every time a keystroke is entered in the textbox, meaning realtime result update (vs. waiting for the Enter key).
Thanks.

Answer 1

Without attaching a profiler and seeing the output I can only guess what is slow.

1. where textBox1.Text.ToLower.Split(' ') can be cached outside the query. I doubt this is a serious issue though.

2. LINQ to XML can be slow on very large XML documents. Consider using XmlTextReader for fast (but ugly) access to underlying XML in a forward-only manner.

3. Preprocess the data in memory if the XML is amenable to that. In memory queries should be pretty fast.

4. You can use something like Lucene.Net to index a large quantity of XML files for quick searching. This might be overkill but it’s pretty fast and scalable.

5. Think about the problem a bit more. Every time the user presses a key you’re processing the XML DOM and doing a bunch of string operations. Processing the XML such that you have a form more suitable to your task seems like a good win.

Note that the response time for autocomplete of this nature is pretty lax, up to a second of lag or so is pretty acceptable for most purposes.