Question

I have a web UI that tries to spawn a MR job on HBase table. I keep getting this error though:

java.io.FileNotFoundException: File /opt/hadoop/mapreduce/system/job_201205251929_0007/libjars/zookeeper-3.3.2.jar does not exist.
        at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:361)
        at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:245)
        at org.apache.hadoop.filecache.DistributedCache.getTimestamp(DistributedCache.java:509)
        at org.apache.hadoop.mapred.JobClient.configureCommandLineOptions(JobClient.java:629)
        at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:761)
        at org.apache.hadoop.mapreduce.Job.submit(Job.java:432)
        at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:448)

I am running with hbase 0.90.4. HBase manages its own zookeeper. And, I confirmed that I have /opt/hadoop/mapreduce/system/job_201205251929_0007/libjars/zookeeper-3.3.2.jar in my HDFS. Is it looking in Local FS?

Answer 1

I found that I did not have core-site.xml on my classpath and it was taking local FS for fs.default.name instead of the HDFS. The jar existed in HDFS but it was looking in local FS.