Home/ Questions/Q 9127145
In Process

The Archive Base Latest Questions

Editorial Team
  • 0
Asked: 2026-06-17T07:12:56+00:00

I have an ouput from my mapper: Mapper: KEY, VALUE(Timestamp, someOtherAttrbibutes) My Reducer does

  • 0

I have an ouput from my mapper:

Mapper: KEY, VALUE(Timestamp, someOtherAttrbibutes)

My Reducer does recieve:

Reducer: KEY, Iterable<VALUE(Timestamp, someOtherAttrbibutes)>

I want Iterable<VALUE(Timestamp, someOtherAttrbibutes)> to ordered by Timestamp attribute. Is there any possibility to implement it?

I would like to avoid manual sorting inside Reducer code. http://cornercases.wordpress.com/2011/08/18/hadoop-object-reuse-pitfall-all-my-reducer-values-are-the-same/

I’ll have to “deep-copy” all objects from Iterable and it can cause huge memory overhead. :(((

Share

Leave an answer

You must login to add an answer.

Need An Account,

1 Answer

  1. Editorial Team

    It’s relatively easy, you need to write comparator class for your VALUE class.

    Take a closer look here: http://vangjee.wordpress.com/2012/03/20/secondary-sorting-aka-sorting-values-in-hadoops-mapreduce-programming-paradigm/ especially at A solution for secondary sorting part.

    • 0