Question

I have a huge file which has two columns per line separated by tab.

I have another file that has a list of values, one in each line.

Now I want to filter the first file, such that I get all lines whose first column is present in the second file.

How to do it in Pig?

Answer 1

You could use an inner join:

A = LOAD 'file1' USING PigStorage('\t') AS (f1:int, f2:int);
B = LOAD 'file2' USING PigStorage(',') AS (f3:int);

C = JOIN A BY f1, B BY f3;
DUMP C;