Question

I am hoping to leverage the unix sort command to sort a large text file in Java. I’ve tried executing sort with the process builder, but with no luck. However when I print the exact command it is going to execute and copy and paste it into the terminal, it works fine.

So far I’ve tried executing with /bin/sh -c “”, making sure the directory the input file is and where the output file will be is fully permissioned (chmod 777) but with no luck.

Here is the code (if it looks funny, note is using some functions found in Guava)

File inputFile = new File(inputFileName);

//build the command (optional number of sort columns)
List<String> command = new LinkedList<String>();
command.addAll(ImmutableList.<String>of("sort","-t"+delimiter));
for (int i : sortFieldPositions) {
    command.add("-k"+i+","+i);
}
command.addAll(ImmutableList.<String>of(inputFileName,">",outputFileName));

//for debugging: output the command that will be executed
System.out.println("Executing: "+Joiner.on(" ").join(command));

//construct and start the process
Process process = new ProcessBuilder(command).redirectErrorStream(true).directory(inputFile.getParentFile()).start();
//for debugging: save process output
BufferedReader bufferedReader = new BufferedReader(new InputStreamReader(process.getInputStream()));
StringBuilder outputStringBuilder = new StringBuilder();
for (String line; (line = bufferedReader.readLine()) != null; /*reading taking place in check */) {
    System.out.println("FROM PROCESS: "+line);
    outputStringBuilder.append(line);
}
bufferedReader.close();

if (process.exitValue() != 0) {
    //something went wrong
    throw new RuntimeException("Error code "+process.exitValue()+" executing command: "+Joiner.on(" ").join(command)+"\n"+outputStringBuilder.toString());
}

Unfortunately this does not work, with the following output:

Executing: sort -t, -k2,2 -k1,1 /tmp/java/TestDataSorterImporterInput.txt /tmp/java/TestDataSorterImporterOutput.txt
FROM PROCESS: sort: stat failed: >: No such file or directory

Edit: It may be helpful to note that if I remove saving the output (> outputfile) from the command, then the command executes without complaint and the sorted version appears in the output from the Processes’ input stream)

Answer 1

It is the shell that knows how to perform output redirection. The sort program cannot do it on its own. So if you want redirection, you need to do /bin/sh -c ... to let she shell into the loop.

(You write that you have tried this, but something else must have gone wrong with that).