For my Hadoop experiments I’ve setup single node HDFS on localhost on my macbook.
Everytime after my experiments I shutdown hadoop services (./bin/stop-all.sh).
Almost every time when I start my setup again (start-all.sh), I find my HDFS corrupted, I always need to format my namenode and restart services to get access of HDFS again.
I realized this as I stop getting namenode admin access (http://localhost:50070/) after startup.
During startup my namenode logs says
“2012-11-29 15:27:43,160 ERROR org.apache.hadoop.hdfs.server.namenode.FSNamesystem: FSNamesystem initialization failed.
org.apache.hadoop.hdfs.server.common.InconsistentFSStateException: Directory /private/tmp/hadoop-rupadhyay/dfs/name is in an inconsistent state: storage directory does not exist or is not accessible.”
I am not sure what I am missing here.
Using a tmp directory means that you have to be sure to clear the directory and reformat your name node every time you restart. This is because the OS may delete files at whim from the tmp directory leaving your hdfs in an unstable state.