Question

Since bz2 is splittable and saves space, I’m planning to store the table data in bzip2 format.
However it seems it’s not recognizing the compressed format. It was fine with gzip format.
The code I used looks like this:

CREATE TABLE IF NOT EXISTS image_bzip (
    image_id STRING,
    image_feature STRING, 
    other_stuff STRING)
ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' STORED AS TEXTFILE LOCATION '/hivestore/bzip';

LOAD DATA LOCAL INPATH '/nfs/directory/*.bz2' OVERWRITE INTO TABLE image_bzip;

Can somebody point me what I’m missing ?

Thanks.

Answer 1

if we set the compression codec correctly, it works naturally.