Question

I have a file on HDFS into which I have written ints in my code using FSDataOutputStream.writeInt()

How do I read it from command line hadoop tool? hadoop dfs -cat returns me binary garbage.

Answer 1

Straight from the command line using some concoction of hadoop fs – unlikely unless you want to delve into piping the output via an awk script.

You could just write a simple java class to read the file, i’m guessing something like:

public class IntFileReader extends Configured implements Tool {
    public static void main(String[] args) throws Exception {
        ToolRunner.run(new IntFileReader(), args);
    }

    public int run(String[] args) throws Exception {
        FileSystem fs = FileSystem.get(getConf());

        FSDataInputStream is = fs.open(new Path(args[0]));

        while (is.available() != -1) {
            System.out.println(is.readInt());
        }

        is.close();

        return 0;
    }
}

Then you can bundle in a jar and execute:

hadoop jar myJar.jar IntFileReader /path/to/file/in/hdfs