Question

I am wondering if anyone can help me out with parsing out data for key words.

say I am looking for this keyword: My Example Yo (this is one of many keywords)

I have data like this

MY EXAMPLE YO #108
my-example-yo #108 
my-example #108 
MY Example #108 

This is just a few combinations. There could be words or number is front of these sentences, there could be in any case, maybe nothing comes after it maybe like the above example something comes after it.

A few ideas came to mind.

1. store all combinations that I can possible think of in my database then use contains

The downside with this is I going a huge database table with every combination of everything thing I need to find. I then will have to load the data into memory(through nhibernate and check every combination). I am trying to determine what category to use based on keyword and they can upload thousands of rows to check for.

Even if I load subsets and look through them I still picture this will be slow.

1. Remove all special characters and make single spaces and ignore case and try to use regex to see how much of the keyword matches up.

Not sure what to do if the keyword has special characters like dashes and such.

I know I will not get every combination out there but I want to try get as many as I can.

Answer 1

Have you considered Lucene.Net? I haven’t used it myself, but I hear it’s a great tool for full text searching. It might do well with keyword searching too. I believe that stackoverflow uses Lucene.