Question

I am new to hadoop. I have mutiple folders containing files to processing a data in hadoop. I have doubt to implement mapper in map-reducer algorithm. Can I specify multiple mappers for processing mulitple files and have all input files as one output using a single reducer? If possible, please give guidelines for implementing the above steps.

Answer 1

If you have multiple files, use MultipleInputs

addInputPath() method can be used to:

1. add multiple paths and one common mapper implementation
2. add multiple paths with custom mapper and input format implementation.

For having a single reducer, have each maps’ output key same…say 1 or “abc”. This way, the framework will create only one reducer.