Question

I forgot what API/method to call, but my problem is that :

My mapper will run more than 10 minutes – and I don’t want to increase default timeout.

Rather I want to have my mapper send out update ping to task tracker, when it is in the particular code path that consumes time > 10 mins.

Please let me know what API/method to call.

Answer 1

You can simply increase a counter and call progress. This will ensure that the task sends a heartbeat back to the tasktracker to know if its alive.

In the new API this is managed through the context, see here: http://hadoop.apache.org/common/docs/r1.0.0/api/index.html

e.G.

@Override
protected void map(LongWritable key, Text value, Context context)
  throws IOException, InterruptedException {
    // increment counter
    context.getCounter(SOME_ENUM).increment(1);
    context.progress();
}

In the old API there is the reporter class:
http://hadoop.apache.org/common/docs/current/api/org/apache/hadoop/mapred/Reporter.html