Question

I’m trying to write and use a custom filter for Solr. The parent application is a Rails app using the Sunspot gem.

I’ve got a filter factory in myorg/solr/analysis/TestThingFilterFactory.java:

package myorg.solr.analysis;

import org.apache.lucene.analysis.TokenStream;
import org.apache.solr.analysis.BaseTokenFilterFactory;
import myorg.solr.analysis.TestThingFilter;

public class TestThingFilterFactory extends BaseTokenFilterFactory {
  public TestThingFilter create(TokenStream input) {
    return new TestThingFilter(input);
  }
}

and a filter in myorg/solr/analysis/TestThingFilter.java:

package myorg.solr.analysis;

import java.io.IOException;
import org.apache.lucene.analysis.Token;
import org.apache.lucene.analysis.TokenFilter;
import org.apache.lucene.analysis.TokenStream;
import org.apache.lucene.analysis.tokenattributes.CharTermAttribute;

public class TestThingFilter extends TokenFilter {
  public TestThingFilter(TokenStream input) {
    super(input);
  }

  public boolean incrementToken() throws IOException {
    // ...
  }
}

I compiled these files with javac -classpath apache-solr-core-3.2.0.jar:lucene-core-3.2.0.jar myorg/solr/analysis/*.java, then made a .jar file from the .class files and put the .jar file in Sunspot’s solr/lib/ directory. I modified Solr’s schema.xml to include the new filter:

<fieldType name="text" class="solr.TextField" omitNorms="false">
  <analyzer>
    <tokenizer class="solr.WhitespaceTokenizerFactory"/>
    <filter class="solr.ASCIIFoldingFilterFactory"/>
    <filter class="solr.LowerCaseFilterFactory"/>
    <filter class="myorg.solr.analysis.TestThingFilterFactory"/>
  </analyzer>
</fieldType>

Restarting Solr and trying to reindex produces this error in the logs:

SEVERE: java.lang.NoClassDefFoundError: org/apache/solr/analysis/BaseTokenFilterFactory
...
Caused by: java.lang.ClassNotFoundException: org.apache.solr.analysis.BaseTokenFilterFactory
...

This is a problem with how I compiled the new filter code, right? How do I compile so it can find the right classes at runtime?

Answer 1

Found the solution: the new .jar file containing the custom analysis code should go in the solr/lib/ directory in the Rails root directory, not within the vendored Sunspot gem. This is the same solr/ directory that houses the conf/ directory.