Question

I’m implementing a classical map-reduce program in which I have a parent that spwans N children(maps) + 1(reduce). The parent sends info, through unnamed pipes, to each one of the N children. The maps process the request and send the result, an int, to reduce. The reduce does a select and sums up every counter writen on the pipes from map to reduce.

At the end, reduce has to send a signal SIGUSR1 with the result, but my code does it many times and wrong, because it prints always o in the signal handler. Is part of the code:

void reduce() {

    int answer;
    int i = 0;
    fd_set set;
    FD_ZERO(&set); //clean set of pipes

    while (1) {
        for (i = 0; i < maps_nr; i++) {
            FD_SET(fd_maps_to_reduce[i][READ], &set); 
        }
        if (select(FD_SETSIZE, &set, NULL, NULL, NULL) > 0) {
            printf("Entrou no select\n");
            for (i = 0; i < maps_nr; i++) { 
                if (FD_ISSET(fd_maps_to_reduce[i][READ], &set)) {
                    close(fd_maps_to_reduce[i][WRITE]);
                    if (read(fd_maps_to_reduce[i][READ], &answer, sizeof (int))) {
                        result += answer;
                        printf("Result in reduce =%d\n", result);
                    } else {
                        printf("Reduce failed to read from pipe from son :%d!\n", i);
                    }
                }
            }
        }//end of select
        printf("Reduce is going to send a signal with result= %d!\n", result);
        kill(getppid(), SIGUSR1);
        printf("Already send!\n");
    }
}

and in the parent, after creating pipes and children I have something like this:

(...)
signal(SIGUSR1, handle_signal);
while(exit) {
    (...)//this is a menu
    for i->N 
        send a struct to each child (through write in respective pipe)
    after the for do:
    pause();//waiting for a signal to be caught
    if (errno==EINTR)
       printf("caught sigusr1");
}

void handle_signal(int signum) {
    signal(SIGUSR1, handle_signal);
    //print results
    printf("Result: %d\n",result);
}

The problem is that the reduce process sums correctly and prints correctly, but the signal is being send many times and I only want one, i.e., in the wend sends a signal sigusr1 to the parent, which is blocked in pause(), and prints the global var result.

How can I do that? Is something wrong in reduce isn’t it?

Answer 1

First, you can create a better-looking select() loop like this:

while (newfds = readfds, select(n, &newfds, NULL, NULL, NULL))

Now, on to your problem. As I see from the code above, you’re signaling the parent every time select() unblocks, which can happen more than once per map process. select() may unblock and run all the rest of the code in the loop every time any of your map processes sends data to the reduce process. Even if it’s half an answer.

If you want to send the signal once you’ve reduced everything, you have to implement some logic to detect that all processes are done, end the loop, and then (outside the loop) signal the parent.

Edit: try something like this (I removed some details of your code, so as to make the example clearer).

void reduce() {

    int i, answer, waiting, ret;
    fd_set read_set, selected_set;

    FD_ZERO(&read_set);

    for (i = 0; i < maps_nr; i++)
        FD_SET(fd_maps_to_reduce[i][READ], &read_set); 

    waiting = maps_nr; /* how many answers are we expecting? */

    while(waiting > 0 &&
          selected_set = read_set,
          select(FD_SETSIZE, &selected_set, NULL, NULL, NULL)) {

        for (i = 0; i < maps_nr; i++) {

            if (FD_ISSET(fd_maps_to_reduce[i][READ], &set)) {
                close(fd_maps_to_reduce[i][WRITE]);

                /* read your result. Once you have it: */
                FD_CLR(fd_maps_to_reduce[i][READ], &read_set);
                /* Now you won't wait for that pipe to produce data. */
                waiting--;
            }

        }
    }

    /* Now you are out of the select loop. Signal, or whatever. */

}

edit 2: and by the way, your result may be printing 0 because you’re dealing with different processes here. The reduce process has its own copy of the result variable, it won’t change the one on the main process. You have to IPC it, maybe another pipe if already wrote code for that.