Question

This wasn’t a problem in 0.9.2, but in 0.10, when I try to access a key in a map that has a dollar sign in it, I get hammered with errors that I haven’t defined the variable. Specifically:

blah = FOREACH meh GENERATE source, json_post_id#'$id' AS post_id;

returns

Undefined parameter : id

That’s fine and makes sense, but when I amend it to:

blah = FOREACH meh GENERATE source, json_post_id#'\$id' AS post_id;

I get:

Unexpected character '$'

Ideas?

[Edit] Forgot to mention: have tried with 2 backslashes and 3 backslashes as well. No dice. [/Edit]

Answer 1

1. Based on a response to your Mail Archive Posting, it looks like the behavior will be “different when using Grunt shell and running it as a script.”

   Input file

   cheolsoo@localhost:~/workspace/pig-svn $cat 1.txt $id,a
   

   Grunt shell

   The $ with no backslash works:

   grunt> A = LOAD '1.txt' USING PigStorage(',') AS (k:chararray,
   v:chararray); grunt> B = FOREACH A GENERATE TOMAP(k, v) AS M; grunt> C
   = FOREACH B GENERATE M#'$id'; grunt> DUMP C; (a)
   

   Script

   The $ with a single backslash works:

   cheolsoo@localhost:~/workspace/pig-svn $cat test.pig A = LOAD '1.txt' 
   USING PigStorage(',') AS (k:chararray, v:chararray); B = FOREACH A
   GENERATE TOMAP(k, v) AS M; C = FOREACH B GENERATE M#'\$id'; DUMP C;
   
   cheolsoo@localhost:~/workspace/pig-svn $./bin/pig -x local test.pig
   (a)
   

2. Also, from Pig problem with split string(STRSPLIT), have you tried either of the following.

   • Correctly escaping the character u0024. Test single using single or double quotes to see if that makes a difference. This answer shows that the single quote makes a difference which you have but it’s worth mentioning.

   • Alternatively, although related, break the loop into a block.

     blah = FOREACH meh {
         GENERATE source, json_post_id#'$id' AS post_id; 
     }
     

3. It looks like you are concatenating an id. Make sure you are not suppose to be using CONCAT(). http://pig.apache.org/docs/r0.10.0/func.html#concat

4. According to the Class PigStorage documentation (Pig 0.10.0 API):

   A load function that parses a line of input into fields using a character delimiter. The default delimiter is a tab. You can specify any character as a literal (“a”), a known escape character (“\t”), or a dec or hex value (“\u001”, “\x0A”).