I’m working on a search engine. For the most part, I’m simply using Appache’s Lucene, which is working great so far, but I also wanted to improve the search results by establishing good “heuristics” within the search. (For example, if someone searches ‘couch’ and I have all of the couches cataloged as type ‘sofa’, I want the search algorithm to make the connection.)
I know this sounds a bit vague, but I don’t know where to continue searching to find further reading in this study. (I Googled terms like ‘heuristic search’, ‘heuristic function’, etc, but they’re not referring to the same thing I am.) So, I wanted to know if any of you guys worked on similar problems in search engines, and if you would recommend anything.
I had to build something similar for my Artificial Intelligence class. I build a web crawler that associated synonyms of words similar to what your looking to do. When a user searches for a term such as ‘couch’, I grabbed all of the synonyms of couch and stored them in a database with a reference to the original word. When the engine gets run again and ‘sofa’ gets searched, the application will again grab synonyms of ‘sofa’ (which is a synonym of couch). You should then be able to match that association.
There are plenty of free api’s to get the synonym of a word. Try changing your google searches to Topic Specific Web Crawlers, or Topic specific search engines. You will gather better results