Sign Up to our social questions and Answers Engine to ask questions, answer people’s questions, and connect with other people.
Login to our social questions & Answers Engine to ask questions answer people’s questions & connect with other people.
Lost your password? Please enter your email address. You will receive a link and will create a new password via email.
Please briefly explain why you feel this question should be reported.
Please briefly explain why you feel this answer should be reported.
Please briefly explain why you feel this user should be reported.
Does anyone know any Apache pig documentation which list down all the operators (like group by, streaming, etc) and the corresponding action taken by PIG i.e what kind/count of MR job(s) the operator results in?
I am specifically interested in streaming aspect, how does it maps to MR job(s).
However far not a complete list, but I think it’s worth reading the following articles/sections:
Building a High-Level Dataflow System on top of Map-Reduce: The Pig Experience
(Section 4. Compilation to MapReduce)
http://infolab.stanford.edu/~olston/publications/vldb09.pdf
Pig Latin: A Not-So-Foreign Language for Data Processing
(Chapter 4.2 Map-Reduce Plan Compilation)
http://infolab.stanford.edu/~olston/publications/sigmod08.pdf
Furthermore you can always issue EXPLAIN or ILLUSTRATE on your script
to see what happens behind the scenes.