Question

I am following this tutorial http://www.michael-noll.com/tutorials/running-hadoop-on-ubuntu-linux-single-node-cluster/

note: yes I know I did install hadoop to /usr/local/hadoop/hadoop/ but the tutorial does not

When I run:

hduser@ubuntu:~$ /usr/local/hadoop/hadoop/bin/hadoop namenode -format

I get

Warning: $HADOOP_HOME is deprecated.

/usr/local/hadoop/hadoop/bin/hadoop: line 320: /usr/lib/jvm/java-6-sun/bin/java: No such  file or directory
/usr/local/hadoop/hadoop/bin/hadoop: line 390: /usr/lib/jvm/java-6-sun/bin/java: No such file or directory

instead of

hduser@ubuntu:/usr/local/hadoop$ hadoop/bin/hadoop namenode -format
10/05/08 16:59:56 INFO namenode.NameNode: STARTUP_MSG:
/************************************************************
STARTUP_MSG: Starting NameNode
STARTUP_MSG:   host = ubuntu/127.0.1.1
STARTUP_MSG:   args = [-format]
STARTUP_MSG:   version = 0.20.2
STARTUP_MSG:   build = https://svn.apache.org/repos/asf/hadoop/common/branches    /branch-0.20 -r 911707; compiled by 'chrisdo' on Fri Feb 19 08:07:34 UTC 2010
************************************************************/
10/05/08 16:59:56 INFO namenode.FSNamesystem: fsOwner=hduser,hadoop
10/05/08 16:59:56 INFO namenode.FSNamesystem: supergroup=supergroup
10/05/08 16:59:56 INFO namenode.FSNamesystem: isPermissionEnabled=true
10/05/08 16:59:56 INFO common.Storage: Image file of size 96 saved in 0 seconds.
10/05/08 16:59:57 INFO common.Storage: Storage directory .../hadoop-hduser/dfs/name has    been successfully formatted.
10/05/08 16:59:57 INFO namenode.NameNode: SHUTDOWN_MSG:
/************************************************************
SHUTDOWN_MSG: Shutting down NameNode at ubuntu/127.0.1.1
************************************************************/

in the code for /usr/local/hadoop/hadoop/bin/hadoop line 320 is:

JAVA_PLATFORM=`CLASSPATH=${CLASSPATH} ${JAVA} -Xmx32m ${HADOOP_JAVA_PLATFORM_OPTS}           
org.apache.hadoop.util.PlatformName | sed -e "s/ /_/g"`

and line 390 is:

    exec "$JAVA" -Dproc_$COMMAND $JAVA_HEAP_MAX $HADOOP_OPTS -classpath "$CLASSPATH" $CLASS "$@"

Any idea how I can fix this?

Answer 1

I have following file : /usr/lib/hadoop-0.20/bin/hadoop-config.sh (Cloudera installation)

Here I can see that it searches for java in following places:

# attempt to find java
if [ -z "$JAVA_HOME" ]; then
  for candidate in \
    /usr/lib/jvm/java-6-sun \
    /usr/lib/jvm/java-1.6.0-sun-1.6.0.*/jre/ \
    /usr/lib/jvm/java-1.6.0-sun-1.6.0.* \
    /usr/lib/j2sdk1.6-sun \
    /usr/java/jdk1.6* \
    /usr/java/jre1.6* \
    /Library/Java/Home \
    /usr/java/default \
    /usr/lib/jvm/default-java ; do
    if [ -e $candidate/bin/java ]; then
      export JAVA_HOME=$candidate
      break
    fi
  done

Is your JAVA_HOME set correctly? Can you set it manually and then try to run it again.

[Edit: based on comment]

1. To check if JAVA_HOME is set: echo $JAVA_HOME
2. Find out where your jvm is located, usually at: /usr/lib/jvm/java-6-sun/
3. Then to set it. Edit the bashrc and bash_profile: vi ~/.bashrc or vi ~/.bash_profile
4. Add the following: export JAVA_HOME=/usr/lib/jvm/java-6-sun/
5. Note that the path should be based on where you found your jvm.