Question

My wordcount program is giving me the desired output file which has all the words and their occurrences. Now, I have to append this code to support a query such that, when I enter a specific word, then it outputs the corresponding filenames & the output of the wordcount program should be the input to query.

I get these errors – Exception in thread “main” java.lang.ArrayIndexOutOfBoundsException: 2 at org.myorg.InvertedIndex.main(InvertedIndex.java:69) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.jav‌​a:43) at java.lang.reflect.Method.invoke(Method.java:616) at org.apache.hadoop.util.RunJar.main(RunJar.java:156)

Answer 1

Well if you want to input it as a console parameter while starting the job and both you mapper and reducer are in a single file then you can simply make a static String in this class:

private static String myWord;

Then initialize it in the main() method:

myWord = args[2];

And use it in you map() method like this:

if(myWord.equals(word.toString()) {
    output.collect(word, one);
}

This will only emit pairs from the mapper to the reducer and the reducer will sum it up. Should work but I did not test it myself.

@Edit: this is mighty strange. The only thing left you can try (imho) is to do something similar to this WordCount 2.0 example.

As you can notice (a bit below the source code), they start it like this:

$ bin/hadoop jar /usr/joe/wordcount.jar org.myorg.WordCount -Dwordcount.case.sensitive=true /usr/joe/wordcount/input /usr/joe/wordcount/output -skip /user/joe/wordcount/patterns.txt

You can change this to:

$ bin/hadoop jar InvertedIndex.jar org.myorg.InvertedIndex /user/james/input /user/james/output -word hello

And chage the line I suggested (myWord = args[2]) to:

for (int i=0; i < args.length; ++i) {
    if ("-word".equals(args[i])) {
        myWord = args[++i];
    }
}

Guess you can use that example as it does something similar o what you want to achieve (pass something as a parameter and then use it). You can check where they are storing all the data that comes from the parameters (method configure()) etc.