Question

I have create A communicator using the construct MPI_Comm_split(), And then I want to try some send/receive communication both at MPI_Comm_World level and at the new-communicators level. Two communicators are created (one with even-process rank the other with odd-rank).

NOTE: I need to try communication only inside the new communicator that contains even-process-ranks (though the rank of the processes is not the original one, I am still concerned with what those process have in their memory space)

My code is as follows,

    #include<stdio.h>
    #include "mpi.h"
    int main(int arg, char **argv){
    int rank,newrank,newrank2;
    int size, newSize,newSize2;
    int data;
    int recv;
    MPI_Comm newComm, newComm2;
    MPI_Status status, newStatus;
    MPI_Init(&arg,&argv);

    MPI_Comm_size(MPI_COMM_WORLD,&size);
    MPI_Comm_rank(MPI_COMM_WORLD,&rank);    
    //sample data
    data = rank * 100 + 1;

    printf("my rank in the WORLD is: %d\n", rank);
    fflush(stdout); 

    if(rank%2==0){
    MPI_Comm_split(MPI_COMM_WORLD,0,rank,&newComm);
    MPI_Comm_rank(newComm,&newrank);
    MPI_Comm_size(newComm,&newSize);    
    printf("my rank in the newComm1 is : %d\n", newrank);
    }else{      
            MPI_Comm_split(MPI_COMM_WORLD,1,rank,&newComm2);
            MPI_Comm_rank(newComm2,&newrank2);
            MPI_Comm_size(newComm2,&newSize2);        
    printf("my rank in the newComm2 is : %d\n", newrank2);
    }

    //Now try some communication at the newCommunicator and at the WORLD level
    if(rank == 0){
    //sending to process 1 wrt WOLD_Comm
    MPI_Send(&data,1,MPI_INT,1,99,MPI_COMM_WORLD);      
    }
    if(rank==1){
    //Receive from process at WORLD_Comm level. 
    MPI_Recv(&recv,1,MPI_INT,0,99,MPI_COMM_WORLD,&status);
    printf("Received at WORLD Level : %d \n\n",recv);
    }
    if(newSize > 2){
    if(newrank == 2){
        //send to process 1 wrt the newComm
        MPI_Send(&data,1,MPI_INT,1,0,newComm);
    }
    if(newrank==1){
        //Receive from process at newComm level.
        MPI_Recv(&recv,1,MPI_INT,2,0,newComm,&status);
        printf("Received at newComm Level : %d \n\n",recv);
    }
    }
    MPI_Finalize();
    return 0;
    }

The error happens when the I try to send/receive data inside new Communicator. The error makes sense because process that do not belong to the new communicator (odd-rank processes) can not use the new communicator handle (newComm).

So my question is how can I check if a given process belongs to the new communicator or not.
So that I can put this guard before doing any communication.

thanks,

Answer 1

I’d revisit the code doing MPI_Comm_split. From what I can tell, you don’t need two separate variables newComm and newComm2. When you call MPI_Comm_split, you can think of multiple communicators being created globally: one for each value of color. But each process calling MPI_Comm_split only gets a reference to one communicator – the one it belongs to.

The if statement with MPI_Comm_split can be replaced by

MPI_Comm_split(MPI_COMM_WORLD, (rank % 2 == 0), rank, &newComm);
MPI_Comm_rank(newComm, &newrank);
MPI_Comm_size(newComm, &newSize);

Two communicators are created, but newComm refers to a different communicator in different processes.

If the odd-numbered processes don’t need to communicate at all, you can make them supply MPI_UNDEFINED for color. This will make them get a value of MPI_COMM_NULL for newComm:

MPI_Comm_split(MPI_COMM_WORLD, (rank % 2) ? MPI_UNDEFINED : 0, rank, &newComm);

Then your code can check if (newcomm == MPI_COMM_NULL).