I’ve been experimenting with Lucene on Google’s App Engine and have run across index corruption problems when adding records to indexes.
The sure fire way to avoid this is to simply rebuild a new index from scratch. Supposedly Lucene is very good at avoiding index corruption with its file formats, but my experience shows differently.
How are other people doing this? Has anyone else tried to switching to a new directory each time they have a rebuilt index?
Lucene indexes are very hard to corrupt. Which library are you using?
May I suggest you SOLR?