Question

I am having, what I can only guess is a schoolboy error with the classpath while trying to execute the following command (Mahout and Lucene are from Apache):

bin/mahout seq2sparse -i <input-dir> -o <output-dir> -wt tfidf -chunk 50 \
-a org.apache.lucene.analysis.standard.StandardAnalyzer --minSupport 2\ 
--minDF 2 --maxDFPercent 75 --norm 2

This gives me the error:

Exception in thread "main" java.lang.IllegalStateException: java.lang.NoSuchMethodException: org.apache.lucene.analysis.standard.StandardAnalyzer.<init>()
at org.apache.mahout.common.ClassUtils.instantiateAs(ClassUtils.java:68)
at org.apache.mahout.vectorizer.SparseVectorsFromSequenceFiles.run(SparseVectorsFromSequenceFiles.java:204)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
at org.apache.mahout.vectorizer.SparseVectorsFromSequenceFiles.main(SparseVectorsFromSequenceFiles.java:55)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)
at org.apache.mahout.driver.MahoutDriver.main(MahoutDriver.java:188)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
Caused by: java.lang.NoSuchMethodException: org.apache.lucene.analysis.standard.StandardAnalyzer.<init>()
at java.lang.Class.getConstructor0(Class.java:2706)
at java.lang.Class.getConstructor(Class.java:1657)
at org.apache.mahout.common.ClassUtils.instantiateAs(ClassUtils.java:62)
... 16 more

The method expected is part of the Apache Lucene library, it is not my own code.
I have added the Lucene jars to the classpath using the following:
export CLASSPATH=$CLASSPATH:mahout-distribution-0.6/lib/*

As a side note, there is no autocompletion when specifying the class at the “-a” flag. Is this an indication of my jar not being visible? I had this autocompletion on another system.

My machine is running Java(TM) SE Runtime Environment (build 1.6.0_26-b03)

Answer 1

It’s probably happening because of export CLASSPATH=$CLASSPATH:mahout-distribution-0.6/lib/*. The asterisk * will expand to the names of JARs separated by space while colons are expected. You can try this instead to get the separator right:

for jar in mahout-distribution-0.6/lib/*; do
    export CLASSPATH="$CLASSPATH":"$jar"
done