Question

I have a pipeline like pattern communication between MPI processes. where processes Send message to each other as pipeline stages.
The following example shows three processes communicating in such pattern.

    #include "mpi.h"
    #include <stdlib.h>
    #include <stdio.h>

    //declare stage function
    void* testcall(void* d);

    int main(int args, char** argv){
        int rank, size;
        MPI_Status status;
        MPI_Init(&args,&argv);
        MPI_Comm_rank(MPI_COMM_WORLD,&rank);
        MPI_Comm_size(MPI_COMM_WORLD,&size);

        if(rank==0){ 
            int k;  
            int x[3] = {10,11,12};
            void* data = malloc(sizeof(int));             
            for( k=0 ; k< 3;k++){           
                data = &x[k];           
                MPI_Send(data,4,MPI_BYTE,1,0,MPI_COMM_WORLD);   
            }
        }

        if(rank==1){                                
            void* rcv = malloc(sizeof(int));
            void* snd = malloc(sizeof(int));
            int k;
            for( k=0 ; k< 3;k++){
                MPI_Recv(rcv,4,MPI_BYTE,0,0,MPI_COMM_WORLD,&status);              
                snd = testcall(rcv);                        
                int z = *(int *) snd;  
                printf("RCV 1: %d \n",z);
                MPI_Send(&snd,4,MPI_BYTE,2,0,MPI_COMM_WORLD);
            }
        }

        if(rank==2){
            void* rcv2 = malloc(sizeof(int));
            void* snd2 = malloc(sizeof(int));
            int k;
            for( k=0 ; k< 3;k++){
                MPI_Recv(rcv2,4,MPI_BYTE,1,0,MPI_COMM_WORLD,&status);
                snd2 = testcall(rcv2);
                int z = *(int *) snd2;
                printf("RCV 2: %d \n",z);
            }
        }

        MPI_Finalize();
        return 0;
    }

    void* testcall(void* d){
        int z = *(int *) d;
        int k = z * 2;
        void* rslt = malloc(sizeof(int));
        rslt = &k;
        return rslt;
    }

output:

RCV1: 20

RCV1: 22

RCV1: 24

RCV: 2136566600

RCV: 2136566600

RCV: 2136566600

I have one problem with the code though. Send from process 0 to process 1 succeed and give me the correct when I print it.

while Send from process 1 to process 2 seems succeed, but when I try to print it is just an unexpected number (as shown in the output above).

I don’t understand why this two sends behave differently. (both are sending a value pointed by a void pointer. why the second send is erroneous )??

help please.

Answer 1

void* testcall(void* d){
    int z = *(int *) d;
    int k = z * 2;
    void* rslt = malloc(sizeof(int));
    rslt = &k;
    return rslt;
}

This code doesn’t do what you expect. The rslt = &k; line overwrites the value of the rslt pointer with the address of k (and you do this in several other statements). First, this leads to memory leaks, as the address of the memory region, allocated by malloc, is lost. Second, k is an automatic (stack) variable at its location would get used for other purposes once testcall returns. It only works in your case because no other function call is present between testcall() and int z = *(int *) snd;. The correct function should be:

void* testcall(void* d){
    int z = *(int *) d;
    int k = z * 2;
    int* rslt = malloc(sizeof(int));
    *rslt = k;
    return rslt;
}

Then this line from the rank 1 code:

MPI_Send(&snd,4,MPI_BYTE,2,0,MPI_COMM_WORLD);

snd is a pointer itself. You are sending the address of the pointer and that’s why rank 2 prints strange values. The correct statement should read:

MPI_Send(snd,4,MPI_BYTE,2,0,MPI_COMM_WORLD);

Output:

RCV 1: 20
RCV 1: 22
RCV 1: 24
RCV 2: 40
RCV 2: 44
RCV 2: 48