Question

I wrote a java class in order to perform multithreaded tasks, each task running an external process.
The process is in charge of converting “.chp” files into “.txt” files. It is written in C.

This process breaks at one point because it disappears when looking at a “top” in my terminal (probably due to a corrupted chp file). The problem is that the process in my java thread does not return. The “process.waitFor()” seems to go on forever (at least ’til the 12 hours I specified for the ExecutorService.

Am I doing something wrong (not catching an exception?)?
I tried setting a class variable of type String in MyThread and putting an error message in place of throwing a new RuntimeException, then print the String at the end of the main, but the thread code doesn’t reach to this point. It still gets stuck at the waitFor().

Shouldn’t the process terminate once the C program has failed?

The program prints on the terminal (cf: MyThread):

A
B
C

main:

String pathToBin = "/path/to/bin";
List<MyThread> threadList = new ArrayList<MyThread>();

for (File f : folderList) {
    File[] chpFilesInFolder = f.listFiles(new FilenameFilter() {

        @Override
        public boolean accept(File dir, String name) {
            if (name.endsWith(".chp")){
                 return true;
            }else{
                 return false;
            }
        }
    });
    File chpFile = writeChpFiles(chpFilesInFolder);
    String[] cmd = {pathToBin, "--arg1", chpFile, "--out-dir", outputFolder};
    MyThread t = new MyThread(cmd, f, chpFilesInFolder);
    threadList.add(t);
}

ExecutorService threadExecutor = Executors.newFixedThreadPool(4);
for(MyThread th : threadList){
    threadExecutor.execute(th);
}
threadExecutor.shutdown();

try {
    threadExecutor.awaitTermination(12, TimeUnit.HOURS);
} catch (InterruptedException e) {
    e.printStackTrace();
}

MyThread:

class MyThread extends Thread{
    private String[] cmd;
    private File chpFolder;
    private File[] chpFilesInFolder;

    public MyThread(String[] cmd, File chpFolder, File[] chpFilesInFolder){
        this.cmd = cmd;
        this.chpFolder = chpFolder;
        this.chpFilesInFolder = chpFilesInFolder;
    }

    @Override
    public void run() {
        Process process = null;
        try{
            System.err.println("A ");
            ProcessBuilder procBuilder = new ProcessBuilder(cmd);
            procBuilder.redirectErrorStream(true);

            System.err.println("B");
            process = procBuilder.start();

            System.err.println("C");
            process.waitFor();

            System.err.println("D");
            if(process.exitValue()!=0) System.err.println("ERROR !"+process.exitValue());

            System.err.println("E");
        }catch(IOException e){
            e.printStackTrace();
        }catch(InterruptedException e){
            e.printStackTrace();
        }catch(Throwable e){
            e.printStackTrace();
        }finally{
            System.err.println("F");
            if(process!=null) {try { process.destroy();} catch(Exception err) {err.printStackTrace();}}
        }

        File[] txtFilesInFolder = chpFolder.listFiles(new FilenameFilter() {

            @Override
            public boolean accept(File dir, String name) {
                if (name.endsWith(".chp.txt")){
                    return true;
                }else{
                    return false;
                }
            }
        });

        if (txtFilesInFolder.length==chpFilesInFolder.length){
            for (File chp : chpFilesInFolder) {
                chp.delete();
            }
            File logFile = new File(chpFolder, "apt-chp-to-txt.log");
               if (logFile.exists()){
                   logFile.delete();
               }
        }else{
            throw new RuntimeException("CHPs have not all been transformed to TXT in "+chpFolder.getAbsolutePath());
        }

Answer 1

Is it possible that your C program is producing output on stdout? If so, you need to read Process.getOutputStream() before Process.waitFor() returns – see https://bugs.java.com/bugdatabase/view_bug?bug_id=4254231

Alternatively, call your C program that a shell script that redirects stdout.

You can use the jstack command to confirm that the thread is indeed blocked at Process.waitFor().