Question

I’m writing a terminal wrapper for a command-line program in Java, and I spawn the subprocess using ProcessBuilder. To send keystrokes to the subprocess, I just write e.getKeyChar() from the GUI straight to the OutputStream given by proc.getOutputStream(). To receive output from the subprocess, I basically have a while loop that reads from the subprocess’s stdout:

while ((b = br.read()) != -1) {
    System.out.println("Read "+b);
    bb[0] = (byte) b;
    // call an event listener with the read byte
    listener.dataReceived(bb);
}

This works, only if I immediately flush the output on both ends. That is, I have to flush every user input and the subprocess has to flush its own stdout in order for stuff to happen. Otherwise, read() blocks, waiting for data, which is never actually sent (subprocess’ stdout just keeps buffering). How can I get I/O going?

Example terminal subprocess:

#include <stdio.h>

int main() {
    char c;
    while((c = getchar()) != -1) {
        printf("Got: %d\n", c);
        // doesn't work in my Java program if the next line isn't present
        fflush(stdout);
    }
    return 0;
}

I’m running on Ubuntu 10.10 with Sun Java 6.

Answer 1

Many many runtime libraries (e.g., I know that libc does this, and wouldn’t be at all surprised if others do too) will buffer their output by default except when the output is to a terminal. This enormously increases the efficiency of data handling when dealing with many lines (e.g., in a normal pipeline) but when there is only a small amount of information, it hurts a lot. If you have access to the source of the subprocess, it’s definitely best to update the code by turning off buffering or adding flushes.

But that’s not always possible, especially when dealing with third-party code. The best other fix I know of in that case is to use a tool like Expect to trick the subprocess. Internally, Expect knows how to pretend to be a terminal (using ptys on Unix and godawful hacks on Windows) so tricking the other programs into turning off (or at least reducing) their buffering. There is a script – unbuffer – for Expect that makes it focus specifically on this sort of use. (In general it can do a lot more than just dealing with unruly buffering, but it’s the best fix anyway.)