Question

I’m not a Java guy but use Solr for searching, but after search about this issue I couldn’t find out why it is happening.

I have a 30-million-records-index with no sorting and the lightest setup I could do, but I have the following exception after a few queries:

SEVERE: java.lang.OutOfMemoryError: Java heap space at
org.apache.lucene.index.SegmentReader.createFakeNorms(SegmentReader.java:1117)
at
org.apache.lucene.index.SegmentReader.fakeNorms(SegmentReader.java:1125)
at
org.apache.lucene.index.SegmentReader.norms(SegmentReader.java:1140)
at
org.apache.solr.search.SolrIndexReader.norms(SolrIndexReader.java:282)
at
org.apache.lucene.search.TermQuery$TermWeight.scorer(TermQuery.java:72)
at
org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:250)
at org.apache.lucene.search.Searcher.search(Searcher.java:171) at
org.apache.solr.search.SolrIndexSearcher.getDocListNC(SolrIndexSearcher.java:988)
at
org.apache.solr.search.SolrIndexSearcher.getDocListC(SolrIndexSearcher.java:884)
at
org.apache.solr.search.SolrIndexSearcher.search(SolrIndexSearcher.java:341)
at
org.apache.solr.handler.component.QueryComponent.process(QueryComponent.java:182)
at
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:195)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1317) at
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:338)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:241)
at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:293)
at
org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:859)
at
org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:602)
at
org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
at java.lang.Thread.run(Thread.java:679)

Then I restart tomcat and it gets back to work until a few requests for falling down again.

I’m not sorting (even I wish it) and the search os most of times by specific indexed fields (not for all of them).

Could you help me? Thanks in advance 🙂

Answer 1

128 MB seems low for a Solr deployment with a few millions records. You can indeed increase the maximum size of the JVM using -Xmx. The -XX:MinHeapFreeRatio just changes the point at which the heap is resized, but you could also use -Xms with the same value as -Xmx to directly allocate the maximum size and avoid any resizes.

However, you may want to try and determine a more precise value for the heap instead of just throwing more memory blindly, as too much memory can be counter-productive latency-wise, because of the longer pauses during garbage collection. Using JVisualVM (even better, with the VisualGC plugin) or jstat on the command line, you can see how much memory Solr uses after starting, how much it uses after a request, and generally how its heap varies during your typical usage.

For example, using jstat -gcutil <PID>, you can see how full the young (E, as in Eden) and old (O) generations of the JVM are (the old generation is what you should be looking at, at first). Or using jstat -gc <PID>, you’ll get the values instead of a percentage (the C columns being the capacity, i.e. the maximum, and the U columns being the actual usage). You need enough memory for Solr’s working set plus what’s needed to process the requests. Using that information, you can tune a bit more finely what’s needed.