Sign Up to our social questions and Answers Engine to ask questions, answer people’s questions, and connect with other people.
Login to our social questions & Answers Engine to ask questions answer people’s questions & connect with other people.
Lost your password? Please enter your email address. You will receive a link and will create a new password via email.
Please briefly explain why you feel this question should be reported.
Please briefly explain why you feel this answer should be reported.
Please briefly explain why you feel this user should be reported.
My mapper emits
‘uniq key’ – ‘very large value’ pair.
My reducer doesn’t know the key is unique.
Thus, the reducer waits until all the mappers are completed.
I tried to use a combiner, but it is not an easy solution for me, because my reducer is very complicated.
My question is how can I perform the reducer after per map? without using a combiner.
If your keys are uniq then the is no need to reduce them. Therefore just copy-paste reducer code to mapper and set reducer number to zero. BTW there are many map reduce jobs which do not require reduce step so it is not something strange.