Home/ Questions/Q 7723123
In Process

The Archive Base Latest Questions

Editorial Team
  • 0
Asked: 2026-06-01T04:21:51+00:00

I am trying to extract some part of string and store it to hbase

  • 0

I am trying to extract some part of string and store it to hbase in columns.

Files Content :

msgType1   Person xyz has opened Internet:www.google.com from IP:192.123.123.123 for duration 00:15:00

msgType2   Person xyz denied for opening Internet:202.x.x.x from IP:192.123.123.123 reason:unautheticated 

msgType1    Person xyz has opened Internet:202.x.x.x from IP:192.123.123.123 for duration 00:15:00

pattern of messages corresponding to msgType is fixed. Now i am trying to store person name, destination , source , duration etc in hbase.

I am trying to to wrtie script in PIG to do this task.
But i am stuck at extracting part.(extracting IP or website name from ‘Internet:202.x.x.x’ token inside string).

I tried Regular expression but its not working for me. Regex alway throw this error : 

ERROR 1045: Could not infer the matching function for org.apache.pig.builtin.REGEX_EXTRACT as multiple or none of them fit. Please use an explicit cast.

is there any other way to extract these value and store it to hbase in PIG or other than PIG?

Share

Leave an answer

You must login to add an answer.

Need An Account,

1 Answer

  1. Editorial Team

    How do you use the REGEX_EXTRACT function ? Have you seen the REGEX_EXTRACT_ALL function ? According to the documentation (http://pig.apache.org/docs/r0.9.2/func.html#regex-extract-all), it should be like this :

    test = LOAD 'test.csv' USING org.apache.pig.builtin.PigStorage(',') AS (key:chararray, value:chararray);
test = FOREACH test GENERATE FLATTEN(REGEX_EXTRACT_ALL (value, '(\\S+):(\\S+)')) as (match1:chararray, match2:chararray);
DUMP test;

    My file is like that :

    1,a:b
2,c:d
3,
    • 0