Home/ Questions/Q 6748545
In Process

The Archive Base Latest Questions

Editorial Team
  • 0
Asked: 2026-05-26T12:34:41+00:00

Let’s say I’m using cleanup() functions in Hadoop MapReduce. How would I add a

  • 0

Let’s say I’m using cleanup() functions in Hadoop MapReduce. How would I add a progress tracking mechanism inside it, let’s say in percentage complete, to display it in console?

Share

Leave an answer

You must login to add an answer.

Need An Account,

1 Answer

  1. Editorial Team

    There is no implementation of the cleanup() of the Mapper.java class.

    When a Hadoop job is run from the command prompt, the following is printed on the console.

    11/10/31 18:15:50 INFO mapreduce.Job: map 0% reduce 0%
    11/10/31 18:16:15 INFO mapreduce.Job: map 50% reduce 0%
    11/10/31 18:16:21 INFO mapreduce.Job: map 100% reduce 0%
    11/10/31 18:16:30 INFO mapreduce.Job: map 100% reduce 100%

    The code for the above is in the Job.java class.

      String report = 
    (" map " + StringUtils.formatPercent(mapProgress(), 0)+
        " reduce " + 
        StringUtils.formatPercent(reduceProgress(), 0));
  if (!report.equals(lastReport)) {
    LOG.info(report);
    lastReport = report;
  }

    Mapper.cleanup() code has to be modified to print the progress to the console and jar file built. I don’t think there is OOB support for the cleanup in Hadoop.

    • 0