Home/ Questions/Q 7688195
In Process

The Archive Base Latest Questions

Editorial Team
  • 0
Asked: 2026-05-31T19:51:52+00:00

I have a map reduce job I am trying to run on a relatively

  • 0

I have a map reduce job I am trying to run on a relatively small data set. I keep running into an issue where the reduce job keeps getting stuck at 16%. My task tracker’s log read:

2012-03-21 17:09:23,829 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201203211704_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.16 MB/s)
2012-03-21 17:09:26,865 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201203211704_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.16 MB/s)
2012-03-21 17:09:32,902 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201203211704_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.16 MB/s)
2012-03-21 17:09:38,938 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201203211704_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.16 MB/s)
2012-03-21 17:09:41,973 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201203211704_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.16 MB/s)
2012-03-21 17:09:48,010 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201203211704_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.16 MB/s)
2012-03-21 17:09:51,045 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201203211704_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.16 MB/s)
2012-03-21 17:09:57,086 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201203211704_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.16 MB/s)
2012-03-21 17:10:03,120 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201203211704_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.16 MB/s)
2012-03-21 17:10:06,154 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201203211704_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.16 MB/s)
2012-03-21 17:10:12,198 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201203211704_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.16 MB/s)
2012-03-21 17:10:18,234 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201203211704_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.16 MB/s)
2012-03-21 17:10:21,271 INFO org.apache.hadoop.mapred.TaskTracker:> attempt_201203211704_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.16 MB/s)
2012-03-21 17:10:27,310 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201203211704_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.16 MB/s)
2012-03-21 17:10:33,342 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201203211704_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.16 MB/s)
2012-03-21 17:10:36,374 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201203211704_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.16 MB/s)
2012-03-21 17:10:42,403 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201203211704_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.16 MB/s)
2012-03-21 17:10:48,435 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201203211704_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.16 MB/s)
2012-03-21 17:10:51,462 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201203211704_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.16 MB/s)
2012-03-21 17:10:57,495 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201203211704_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.16 MB/s)
2012-03-21 17:11:03,523 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201203211704_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.16 MB/s)
2012-03-21 17:11:06,545 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201203211704_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.16 MB/s)
2012-03-21 17:11:12,578 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201203211704_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.16 MB/s)
2012-03-21 17:11:18,607 INFO org.apache.hadoop.mapred.TaskTracker: attempt_201203211704_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.16 MB/s)
Share

Leave an answer

You must login to add an answer.

Need An Account,

1 Answer

  1. Editorial Team

    I had opened a FileSystem object and hadn’t closed it in the mapper where files were accesssed. Adding a fs.close() at the end of the map() definition fixed the problem.

    • 0