Question

I had a problem with a program that uses MPI and I have just fixed it, however, I don’t seem to understand what was wrong in the first place. I’m quite green with programming relates stuff, so please be forgiving.

The program is:

#include <iostream>
#include <cstdlib>
#include <mpi.h>

#define RNumber     3

using namespace std;

int main() {
    /*Initiliaze MPI*/
    int     my_rank;    //My process rank
    int         comm_sz;    //Number of processes
    MPI_Comm    GathComm;   //Communicator for MPI_Gather
    MPI_Init(NULL, NULL);
    MPI_Comm_rank(MPI_COMM_WORLD, &my_rank);
    MPI_Comm_size(MPI_COMM_WORLD, &comm_sz);

    /*Initialize an array for results*/
    long rawT[RNumber];
    long * Times = NULL;        //Results from threads
    if (my_rank == 0) Times = (long*) malloc(comm_sz*RNumber*sizeof(long));

    /*Fill rawT with results at threads*/
    for (int i = 0; i < RNumber; i++) {
        rawT[i] = i;
    }


    if (my_rank == 0) {
        /*Main thread recieves data from other threads*/
        MPI_Gather(rawT, RNumber, MPI_LONG, Times, RNumber, MPI_LONG, 0, GathComm);

    }
    else {
        /*Other threads send calculation results to main thread*/
        MPI_Gather(rawT, RNumber, MPI_LONG, Times, RNumber, MPI_LONG, 0, GathComm);
    }

    /*Finalize MPI*/
    MPI_Finalize();
    return 0;
};

On execution the program returns the following message:

Fatal error in PMPI_Gather: Invalid communicator, error stack:
PMPI_Gather(863): MPI_Gather(sbuf=0xbf824b70, scount=3, MPI_LONG,
rbuf=0x98c55d8, rcount=3, MPI_LONG, root=0, comm=0xe61030) failed
PMPI_Gather(757): Invalid communicator Fatal error in PMPI_Gather:
Invalid communicator, error stack: PMPI_Gather(863):
MPI_Gather(sbuf=0xbf938960, scount=3, MPI_LONG, rbuf=(nil), rcount=3,
MPI_LONG, root=0, comm=0xa6e030) failed PMPI_Gather(757): Invalid
communicator

After I remove GathComm altogether and substitute it with MPI_COMM_WORLD default communicator everything works fine.

Could anyone be so kind to explain what was I doing wrong and how did this adjustment made everything work?

Answer 1

That’s because GathComm has not been assigned a valid communicator. “MPI_Comm GathComm;” only declares the variable to hold a communicator but doesn’t create one.

You can use the default communicator (MPI_COMM_WORLD) if you simply want to include all procs in the operation.

Custom communicators are useful when you want to organised your procs in separate groups or when using virtual communication topologies.

To find out more, check out this article which describes Groups, Communicator and Topologies.