Question

I’m thinking about using Lucene in my project to do very fast searches. I know that Lucene creates its own files where it keeps all the data/indexes.

I wonder what are the downsides of using Lucene? Are there any?

Do you have to do anything with the file database or does it work great without any outside help?

P.S. I know there is also Lucene .NET and I bet the same rules apply there.

Answer 1

Lucene is great. Very flexible, surprisingly fast, and a solid API. The mailing list is extremely helpful.

The files do need a bit of maintenance, but it can be done with provided tools. Of primary importance is optimizing the index on occasion, but this is only needed if you update the index regularly.

I would suggest looking into Solr as well. It’s essentially a webapp and tools that sit on top of Lucene. It makes it a tad easier to create new indexes, keep them optimized, as well as providing master/slave synchronization for a scalable search cluster. This, of course, depends on your actual needs.

For a personal example, I used to maintain a search index for a large, well-known, gaming company. The index had hundreds of thousands of entries in multiple languages (world-wide) and locales. It performed a million searches each day on the cluster without using hardly any CPU, and a reasonable amount of memory. It had load tested out to around 300 million searches per day, on the hardware we had and would scale linearly by simply adding more boxes to the cluser. Solr and Lucene were the primary tools for this.

If I had to give a downside, it would be learning curve. There is quite a bit to understand, and if you want a truly optimized solution, you need to know it well. However, this will happen with any search tool you use, if you do it yourself. The documentation, wikis, and mailing list provide plenty of support for this ramp up.