I have an e-commerce website built upon ASP.net MVC3. It has appx. 200k products. We are currently searching in product table for providing search on site.
The issue is it is deadly slow, and of course, by analyzing performance on profiler we found that it is the sql search which is the main issue.
What are other alternatives that could be used to speed up the search? Do we need to build a separate cache for search or anything else is needs to be done?
If you look at the other large e-commerce sites like ebay, amazon or flipkart, they are very fast. How do they do it?
They usually build a full text index of what is searchable, using for example Lucene.NET or Solr (uses Java, but an instance of it can be called through .NET using SolrNet).
The index is built from your SQL database and any searches you do would need to make use of that index by sending queries to it like you would on a SQL database, but with a different syntax of course. The index needs to be updated/recreated periodically to stay up to date with the SQL database.
Such a text index is built for querying large amounts of string data and can easily handle hundreds of thousands of products in a product search function on your website. Aside from speed, there are other benefits that would be very hard to do without a text index, such as spelling corrections and fuzzy searches.