Question

Can anyone help me out on how to print the key and the values that is being used in the reducer code below

I tried to print out values via Logger as well as System.out, but we couldnt see the output on the logger as well as in the console.

 public static class IntSumReducer
       extends Reducer<Text,IntWritable,Text,IntWritable> {
    private IntWritable result = new IntWritable();
    //private static final Log LOG = LogFactory.getLog(WordCount.class);
    public void reduce(Text key, Iterable<IntWritable> values,
                       Context context
                       ) throws IOException, InterruptedException {
      int sum = 0;

      for (IntWritable val : values) {
        sum += val.get();
      //LOG.info("val = " + val.get());
      //System.out.println("val = " + val.get());
      }
      //System.out.println("sum = " + sum + " key = " + key);
      //LOG.info("sum = " + sum + " key = " + key);
      result.set(sum);
      context.write(key, result);
    }
  }

Answer 1

System.out and LOG.info should both work – but you have to go and find the logs for the reduce attempt.

You can find these in the Job Tracker Web UI (http://jobtracker:50030) – locate your job in the list of running, complete, failed or retired jobs, and click on the hyperlinked job ID. This will take you to a view of the Job run. From here you should be able to click on the reducers tasks (there will be a number for pending, running, complete, failed and killed – preferably click on one of the numbers for complete.

Now you will have a list of completed reduce tasks – clicking on a particular task should then list the attempts for that task, and to the right of the table will be a link for the Logs (first 4k, last 4, all), where you should be able to find your output messages (assuming you remove the comments in front of your code above and run your job with the System.out or LOG.info allowed to execute)