Home/ Questions/Q 8608479
In Process

The Archive Base Latest Questions

Editorial Team
  • 0
Asked: 2026-06-12T03:37:18+00:00

I have a mapreduce Mapper. This Mapper should use some set of read-only parameters.

  • 0

I have a mapreduce Mapper. This Mapper should use some set of read-only parameters.
Let’s imagine that I want to count occurences of some substrings (title of something) in input lines.
I do have a list of pairs : “some title” => “a regular expression to extract this title from input line”.
These pairs are stored in usual text file.

What is the best way to pass this file to Mapper?
I have only this idea:

  1. Upload file with pairs to hdfs.
  2. Pass path to file using -Dpath.to.file.with.properties
  3. in static{} section of mapper read file and populate map pair “some title” => “regular expr for the title”.

Is it good or bad? please adivce

Share

Leave an answer

You must login to add an answer.

Need An Account,

1 Answer

  1. Editorial Team

    You’re on track, but I would recommend using the distributed cache. Its purpose is for exactly this – passing read-only files to task nodes.

    1. Put file in HDFS
    2. Add that file to the distributed cache in the main method of your application.
    3. In the Mapper class, override either the configure or setup method depending on which version of the API you are using. In that method it can read from the distributed cache and store everything in memory.
    • 0