Question

I am using Lucene/Solr for my log search tool. Is there a way I can perform a commit operation on my IndexWriter when a particular set of docs is flushed from memory to the disk. My RamBufferSize is 24Mb and MergeFactor is 10.

Or is calling commit in frequent intervals irrespective of the flushes the only way? I wish the autocommit feature was not deprecated.

Answer 1

You might consider using the commitWithin feature of Solr. This will hand off the commit responsibilities to Solr.

CommitWithin is a commit strategy introduced in Solr 1.4, which lets
the client ask Solr to make sure this gets committed within a
certain time. This leaves the control of when to do the commit to Solr
itself, optimizing number of commits to a minimum while still
fulfilling the update latency requirements. If I say (in an XMLUpdateRequestHandler update), that tells
Solr to make sure the document gets committed within 10000ms, i.e.
10s. I can then continue to add other documents within those 10
seconds (possibly with other commitWithin values), and Solr will
automatically do a when the oldest in the buffer is
due.

I use this quite frequently when adding documents to help keep my as close to NRT (near real time) as I can, without the need for issuing commits at frequent intervals.