Question

As an exercise in Unix programming I wrote a program which creates two pipes, forks a child and then sends and receives some text to and from the child via the pipes. It works if in the child process I read and write the data using my code in function filter. However, if the child tries to redirect the pipes to its stdin and stdout (using dup2) and execute (using execlp) the tr utility, then it doesn’t work, it gets stuck somewhere. This code is in the filter2 function. The question is, why? Here is the code:

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <string.h>
#include <ctype.h>

void err_sys(const char* x) { perror(x); exit(1); } 

void upper(char *s) { while((*s = toupper(*s))) ++s; }

void filter(int input, int output)
{   
    char buff[1024];
    bzero(buff, sizeof(buff));
    size_t n = read(input, buff, sizeof(buff));

    printf("process %ld: got '%s'\n", (long) getpid(), buff);

    upper(buff);
    write(output, buff, strlen(buff));
}   

void filter2(int input, int output)
{   
    if (dup2(input, 0) != 0) err_sys("dup2(input, 0)");
    if (dup2(output, 1) != 1) err_sys("dup2(output, 1)");
    execlp("/usr/bin/tr", "tr", "[a-z]", "[A-Z]" , (char*)0);
}   

int main(int argc, char** argv) 
{   
    int pipe1[2];
    int pipe2[2];
    if (pipe(pipe1) < 0) err_sys("pipe1");
    if (pipe(pipe2) < 0) err_sys("pipe2");

    pid_t pid;
    if ((pid = fork()) < 0) err_sys("fork");
    else if (pid > 0)
    {   
        close(pipe1[0]);
        close(pipe2[1]);
        char* s = "Hello there, can you please uppercase this and send it back to me? Thank you!";
        write(pipe1[1], s, strlen(s));

        char buff[1024];
        bzero(buff, sizeof(buff));
        size_t n = read(pipe2[0], buff, sizeof(buff));
        pid_t mypid = getpid();
        printf("process %ld: got '%s'\n", (long) mypid, buff);
    } else
    {   // Child.
        close(pipe1[1]);
        close(pipe2[0]);

        filter(pipe1[0], pipe2[1]); 
        //filter2(pipe1[0], pipe2[1]);  // FIXME: This doesn't work
    }   
    return 0;
} 

Answer 1

Your parent process in main needs a small change:

/* Was: */
char* s = "Hello there, can you please uppercase this and send it back to me? Thank you!";
write(pipe1[1], s, strlen(s));
/* add: */
close(pipe1[1]);

Other people have mentioned buffering, but it’s not really a buffering issue. It’s about interprocess communication.

There is a reason why pipes are called ‘pipes’ and not, say, ‘conveyor belts’. Pipes, unlike conveyor belts, don’t retain package boundaries. A pipe is just a stream of bytes; write dumps a bunch of bytes into the stream, but does not mark the fact that it has done so. Consequently, your code could have identically been:

    write(pipe1[1], s, strlen(s)/2);
    write(pipe1[1], s + strlen(s)/2,
                    strlen(s+strlen(s)/2));

or any other combination of writes. The receiving end will just read a convenient number of bytes (convenient to it, that is), and process them. It probably does something like this:

     read(stdin, buffer, BUFSIZ);

which will not return until either BUFSIZ bytes have been read, or until EOF is reached. Since you cannot reach into the reading process’s system calls and retroactively change the length of the read, the only way you can get the reading process to actually finish its work is to arrange for it to get an EOF indication, and the only way you can do that is to close the pipe. Hence my solution above.

That’s not always convenient, because it makes it impossible to put two consecutive requests into a stream. There is quite a lot of overhead involved in establishing communication between two processes (particularly if the server process needs to be started fresh). If you want to “pipeline” requests (so that responses are sent at the end of every request), you need to design a communications protocol which clearly indicates the “package boundaries”; the division between the requests. In other words, you need to implement your own conveyor belt, using a pipe.

A communications protocol requires support from both ends; you cannot just implement it from the client. So you’re not going to be able to get tr to understand an arbitrary protocol; it just does what it does (reads to EOF and writes translated bytes when it feels it has enough of them to bother sending). So if you want to play around with this idea, you’ll need to write both the client and the server process.

The simplest package protocol available is probably Daniel Bernstein’s netstrings. The link contains actual code, which is delightfully simple, but the basic idea is this: strings are sent by sending their length as a decimal number, followed by a colon (:), followed by exactly the number of bytes promised in the length. The writer needs to know how many bytes it will send before it sends them; the reader needs to read up to the ‘:’ (djb uses scanf to do this, which demonstrates an often underappreciated feature of scanf); once it knows how many bytes are in the request, it can then block read exactly that number of bytes. This is a trivial protocol to implement on both sides, so it makes for a simple practical exercise.

HTTP uses a similar but much more complicated protocol (and, as with all unnecessarily complicated protocols, the result is that interoperability bugs were common because of misunderstanding), but in essence it’s the same: the sender needs to indicate how long the message (or body of the message, in the case of HTTP) is, which it does with a Content-Length: header. However, since it is not always convenient to know how many bytes you’re going to send before you send them all, HTTP allows “chunked” encoding (indicated with a different header); in that case, each chunk consists of a length (in hexadecimal), followed by \r\n followed by the body, followed by \r\n, followed by… well, you can read the RFC for the messy details. Problems here include the fact that some clients send just \n instead of \r\n and that it is slight ambiguous how to handle the trailing \r\n. Netstrings, as djb points out, would have been a lot simpler.

Unless you want to use a full HTTP client/server library, a more practical alternative for implementing interprocess communications is Google’s open-sourced protobuf package. For an earlier and in my opinion technically-superior solution, which unfortunately does not have a convenient set of open-source tools, is ASN.1 (but don’t plunge into that site right away; it’s big).