Question

Calling fclose() here after dup()ing its file descriptor blocks until the child process has ended (presumably because the stream has ended).

FILE *f = popen("./output", "r");
int d = dup(fileno(f));
fclose(f);

However by manually performing the pipe(), fork(), execvp() of the popen(), and then dup()ing the pipe’s read file descriptor, closing the original does not block.

int p[2];
pipe(p);
switch (fork()) {
    case 0: {
        char *argv[] = {"./output", NULL};
        close(p[0]);
        dup2(p[1], 1);
        execvp(*argv, argv);
    }
    default: {
        close(p[1]);
        int d = dup(p[0]);
        close(p[0]);
    }
}

Why does this occur, and how can I close the FILE * returned from popen() and use a file descriptor in its place?

Update:

I’m aware that the documentation says to use pclose(), however fclose() blocks as well. Furthermore, I poked around in the glibc code, and pclose() just calls fclose(). The behaviour is the same, whether fclose() or pclose() is used.

Answer 1

Disappointed with the generality of the answers so far (I can RTFM, tyvm), I’ve investigated this thoroughly, by stepping through and reading the glibc source.

In glibc pclose() directly calls fclose() with no additional effect, so the 2 calls are same. In fact you could use pclose() and fclose() interchangeably. I’m sure this is purely a coincidence in the evolved implementation, and the use of pclose() to close a FILE * returned from popen() is still recommended.

The magic is in popen(). FILE *s in glibc contain a jump table with pointers to appropriate functions to handle such calls as fseek(), fread(), and of relevance fclose(). When calling popen(), a different jump table used than the one used by fopen(). The close member in this jump table points to a special function _IO_new_proc_close, which calls waitpid() on the pid stored in the region pointed to by FILE *.

Here’s the relevant call stack in my version of glibc, which I’ve annotated with notes about what is going on:

// linux waitpid system call interface
#0  0x00f9a422 in __kernel_vsyscall ()
#1  0x00c38513 in __waitpid_nocancel () from /lib/tls/i686/cmov/libc.so.6

// removes fp from a chain of proc files
// and waits for the process of the stored pid to terminate
#2  0x00bff248 in _IO_new_proc_close (fp=0x804b008) at iopopen.c:357

// flushes the stream and calls close in its jump table
#3  0x00c09ff3 in _IO_new_file_close_it (fp=0x804b008) at fileops.c:175

// destroys the FILEs buffers
#4  0x00bfd548 in _IO_new_fclose (fp=0x804b008) at iofclose.c:62

// calls fclose
#5  0x00c017fd in __new_pclose (fp=0x804b008) at pclose.c:43

// calls pclose
#6  0x08048788 in main () at opener.c:34

So the short of it is, using popen(), the returned FILE * must not be closed, even if you dup() its file descriptor, because it will block until the child process terminates. Of course, after this you’ll be left with a file descriptor to a pipe which will contain whatever the child process managed to write() to it before terminating.

By not fread()ing with the file pointer returned from popen(), the underlying pipe will not be touched, it’s safe to use the file descriptor by fileno(), and finish up by calling pclose().