Question

I’ve been struggling with launching a java process from perl. The root of the problem is that the java process is missing the JAVA_HOME environment variable causing a ClassNotFoundException.

I started by using IPC::Run3 because of its relatively elegant redirection of STDIN/STDOUT.

Assuming IPC::Run3 would use %ENV, I tried adding $ENV{JAVA_HOME}.

When that didn’t work I tried doing system(). That didn’t work, so finally, I got it to work using system("JAVA_HOME=/path/to/java && /path/to/java_program");

My test program is below. Naturally I’d uncomment the proper block to test the appropriate invocation.

#!/usr/bin/perl -w
use strict;

use IPC::Run3;

use vars qw(%Config $nutch_stdout $nutch_stderr);

%Config = (
  'nutch_binary'       => q[/home/crawl/nutch/runtime/local/bin/nutch],
  'nutch_crawl_dir'    => q[/home/crawl/nutch-crawl/crawl/crawldb/current/part-00000],
  'nutch_seed_dir'     => q[/home/crawl/urls],
  'solr_url'           => q[http://localhost:8080/solr],
);

my @nutch_command = ("$Config{nutch_binary}",
                 "crawl $Config{nutch_seed_dir}",
                 "-solr $Config{solr_url}",
                 "-d    $Config{nutch_crawl_dir}",
                 "-threads 1",
                 "-depth 1");

$ENV{JAVA_HOME}       = '/usr/lib/jvm/java-1.6.0';

while ((my $key,my $value) = each %ENV) {
    print "$key=$value\n";
}

print "Running @nutch_command\n";

# My original code. Next few lines are shown in first batch of output below.
#run3 \@nutch_command, undef, \$nutch_stdout, \$nutch_stderr;
#print "Output from Nutch:\n";
#print $nutch_stdout;
#print "Errors from Nutch:\n";
#print $nutch_stderr;

# Second try. The next line's output is the second batch of output.
#system(@nutch_command);

# Third try. Despite setting and displaying %ENV, this is the only thing I tried that worked
system("JAVA_HOME=/usr/lib/jvm/java-1.6.0 && @nutch_command");

Here’s the output of running the run3:

    -bash-3.2$ ./test.pl 
    ... [snip] ...
    JAVA_HOME=/usr/lib/jvm/java-1.6.0
    ... [snip] ...
    Running /home/crawl/nutch/runtime/local/bin/nutch crawl /home/crawl/urls -solr http://localhost:8080/solr -d    /home/crawl/nutch-crawl/crawl/crawldb/current/part-00000 -threads 1 -depth 1
    Output from Nutch:
    Errors from Nutch:
    Exception in thread "main" java.lang.NoClassDefFoundError: crawl
    Caused by: java.lang.ClassNotFoundException: crawl
at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
at java.lang.ClassLoader.loadClass(ClassLoader.java:321)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294)
at java.lang.ClassLoader.loadClass(ClassLoader.java:266)
    Could not find the main class: crawl. Program will exit.

And the output of the first system() call:

    -bash-3.2$ ./test.pl
    ... [snip] ...
    JAVA_HOME=/usr/lib/jvm/java-1.6.0
    ... [snip] ...
    Running /home/crawl/nutch/runtime/local/bin/nutch crawl /home/crawl/urls -solr http://localhost:8080/solr -d    /home/crawl/nutch-crawl/crawl/crawldb/current/part-00000 -threads 1 -depth 1
    Exception in thread "main" java.lang.NoClassDefFoundError: crawl
    Caused by: java.lang.ClassNotFoundException: crawl
at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
at java.lang.ClassLoader.loadClass(ClassLoader.java:321)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294)
at java.lang.ClassLoader.loadClass(ClassLoader.java:266)
    Could not find the main class: crawl. Program will exit.

Finally, the third system call– the only one that worked!– with the environment variable set inline:

    -bash-3.2$ ./test.pl
    ... [snip] ...
    JAVA_HOME=/usr/lib/jvm/java-1.6.0
    ... [snip] ...
    Running /home/crawl/nutch/runtime/local/bin/nutch crawl /home/crawl/urls -solr http://localhost:8080/solr -d    /home/crawl/nutch-crawl/crawl/crawldb/current/part-00000 -threads 1 -depth 1
    crawl started in: crawl-20120216133832
    ... continue success stdout output

Finally to the question: Aside from having to set the environment in-line with the system() call, what’s the appropriate way to pass an environment var to a IPC::Run3 or a system() call?

(Note: output of %ENV is truncated to only relevant lines… lines like PATH, SHELL, _, etc. not relevant to the question omitted)

In case it’s relevant:

-bash-3.2$ uname -a
Linux hostname 2.6.18-238.el5xen #1 SMP Thu Jan 13 16:41:45 EST 2011 x86_64 x86_64 x86_64 GNU/Linux
-bash-3.2$ perl --version
This is perl, v5.8.8 built for x86_64-linux-thread-multi

Answer 1

The root of the problem is that the java process is missing the JAVA_HOME environment variable causing a ClassNotFoundException.

REVISED

That is not the root of the problem. In fact, Java itself does not require JAVA_HOME to be set.

The immediate cause of the problem is one of the following:

• The wrapper is not setting the classpath correctly for the application that you are trying to execute.

• The wrapper using the wrong class name. The class name “nutch” is unusual and suspicious – there’s no package name.

It seems likely that the real root cause is that you are assembling the argument list incorrectly. Each of those arguments with a space inside them should really be two arguments; i.e.

        my @nutch_command = ("$Config{nutch_binary}",
             "crawl", "$Config{nutch_seed_dir}",
             "-solr", "$Config{solr_url}",
             "-d", "$Config{nutch_crawl_dir}",
             "-threads", "1",
             "-depth", "1");

I suspect that this has confused the nutch wrapper script, and caused it to use the wrong classname (among other things). When you pass the entire command as one string and let the shell parse it, the problem (naturally) goes away.