Question

I am running a hadoop code that has a partitioner class inside the job. But, when I run the command

hadoop jar Sort.jar SecondarySort inputdir outputdir

I am getting a runtime error that says

class KeyPartitioner not org.apache.hadoop.mapred.Partitioner.

I have ensured that the KeyPartitioner class has extended the Partitioner class, but why am I getting this error?

Here is the driver code:

JobConf conf = new JobConf(getConf(), SecondarySort.class);
    conf.setJobName(SecondarySort.class.getName());

    conf.setJarByClass(SecondarySort.class);

    conf.setInputFormat(TextInputFormat.class);
    conf.setOutputFormat(TextOutputFormat.class);

    conf.setMapOutputKeyClass(StockKey.class);
    conf.setMapOutputValueClass(Text.class);

    conf.setPartitionerClass((Class<? extends Partitioner<StockKey, DoubleWritable>>) KeyPartitioner.class);

    conf.setMapperClass((Class<? extends Mapper<LongWritable, Text, StockKey, DoubleWritable>>) StockMapper.class);
    conf.setReducerClass((Class<? extends Reducer<StockKey, DoubleWritable, Text, Text>>) StockReducer.class);

and here is the code of the partitioner class:

public class KeyPartitioner extends Partitioner<StockKey, Text> {

@Override
public int getPartition(StockKey arg0, Text arg1, int arg2) {

    int partition = arg0.name.hashCode() % arg2;

    return partition;
}
}  

Answer 1

Notice that there are two partitioners in hadoop:

org.apache.hadoop.mapreduce.Partitioner
org.apache.hadoop.mapred.Partitioner

Make sure your KeyPartitioner class implements the second interface, not the first abstract class.

Edit: You have to set the input and output folders:

FileInputFormat.addInputPath(conf, new Path(args[0]));
FileOutputFormat.setOutputPath(conf, new Path(args[1]));