Question

I have locally installed Hadoop on Debian linux distribution in vmware and I am having difficulties running my test jobs.

I can easily build create the package with IntelliJ Idea, even my mrunit tests pass as expected.

Once I try to run the job, I get following exception:

Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/tools/ant/launch/AntMain
    at java.lang.ClassLoader.defineClass1(Native Method)
    at java.lang.ClassLoader.defineClass(ClassLoader.java:791)
    at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
    at java.net.URLClassLoader.defineClass(URLClassLoader.java:449)
    at java.net.URLClassLoader.access$100(URLClassLoader.java:71)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:423)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:356)
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:264)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:190)
Caused by: java.lang.ClassNotFoundException: org.apache.tools.ant.launch.AntMain
    at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:423)
at java.lang.ClassLoader.loadClass(ClassLoader.java:356)

I made sure, all the dependencies, are included in the jar – I can see, that ANT 1.6.5 is included.

I even tried to install ANT and set $ANT_HOME to its location, but with no effect.

What I might be doing wrong?

I am using following pom.xml:

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <groupId>tl.knyt</groupId>
    <artifactId>hanoi</artifactId>
    <version>1.0</version>
    <name>Hanoi</name>
    <url>http://maven.apache.org</url>

    <dependencies>
        <dependency>
            <groupId>org.apache.hadoop</groupId>
            <artifactId>hadoop-core</artifactId>
            <version>0.20.2-cdh3u4</version>
        </dependency>

        <dependency>
            <groupId>org.apache.hadoop</groupId>
            <artifactId>hadoop-mrunit</artifactId>
            <version>0.20.2-cdh3u4</version>
        </dependency>

        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <version>4.10</version>
        </dependency>

    </dependencies>
    <repositories>
        <repository>
            <id>cloudera.repos</id>
            <url>https://repository.cloudera.com/artifactory/cloudera-repos/</url>
            <name>Cloudera Repos</name>
            <snapshots>
                <enabled>true</enabled>
            </snapshots>
        </repository>
    </repositories>
</project>

Thanks!

Answer 1

Make sure in your hadoop classpath you have the jar file for org/apache/tools/ant/launch/AntMain

you can edit the jar file in $HADOOP_HOME/conf/hadoop-site file

you’ll have to restart hadoop for the change to take effect.