Question

I was wondering why this program actually works in MPI (openMPI 1.5/1.6. )

#include <stdio.h>
#include <mpi.h>

#define VECTOR_SIZE 100

int main(int argc,char ** argv) {
  int A[VECTOR_SIZE];
  int sub_size=2;
  int count=10;
  MPI_Datatype partial_array;
  int rank,size;
  MPI_Status status;

  MPI_Init(&argc,&argv);
  MPI_Comm_rank(MPI_COMM_WORLD,&rank);
  MPI_Comm_size(MPI_COMM_WORLD,&size);

  MPI_Type_vector(count, sub_size,
          2*sub_size, MPI_INT, &partial_array);

  MPI_Type_commit(&partial_array);

  if (rank == 0) {
    int i;
    // server - initialize data and send
    for (i = 0; i< VECTOR_SIZE; i++) {   
      A[i] = i;
    }
    MPI_Send(&(A[0]), 1, partial_array, 1, 0, MPI_COMM_WORLD);
  } else if (rank==1) {
    int i;
    for (i = 0; i< VECTOR_SIZE; i++) {

        A[i] = 0;

    }
    // vector is composed by 20 MPI_INT elements
    MPI_Recv(&(A[0]),20, MPI_INT, 0, 0, MPI_COMM_WORLD, &status);

    printf("\n");

    for (i = 0; i<VECTOR_SIZE; i++) {
      printf("%d ",A[i]);

    }
    printf("\n");
  }

  MPI_Finalize();

}

while this other program where Send and Receive primitives are exchanged does not terminate (the receive never completes):

#include <stdio.h>
#include <mpi.h>

#define VECTOR_SIZE 100

int main(int argc,char ** argv) {
    int A[VECTOR_SIZE];
    int sub_size=2;
    int count=10;
    MPI_Datatype partial_array;
    int rank,size;
    MPI_Status status;

    MPI_Init(&argc,&argv);
    MPI_Comm_rank(MPI_COMM_WORLD,&rank);
    MPI_Comm_size(MPI_COMM_WORLD,&size);

    MPI_Type_vector(count, sub_size,
                    2*sub_size, MPI_INT, &partial_array);

    MPI_Type_commit(&partial_array);

    if (rank == 0) {
        int i;
        // server - initialize data and send
        for (i = 0; i< VECTOR_SIZE; i++) {

            A[i] = i;

        }
        MPI_Send(&(A[0]),20, MPI_INT, 0, 0, MPI_COMM_WORLD);

    } else if (rank==1) {
        int i;
        // client - receive data and print
        for (i = 0; i< VECTOR_SIZE; i++) {

            A[i] = 0;

        }

        MPI_Recv(&(A[0]), 1, partial_array, 1, 0, MPI_COMM_WORLD, &status);

        printf("\n");

        for (i = 0; i<VECTOR_SIZE; i++) {
            printf("%d ",A[i]);

        }
        printf("\n");
    }

    MPI_Finalize();

}

If I understand MPI type mathing rules correctly neither of the two should complete.

Answer 1

Obviously in the second program rank 0 is sending to itself and rank 1 is expecting message also from itself:

MPI_Send(&(A[0]),20, MPI_INT, 0, 0, MPI_COMM_WORLD);

destination rank should be 1, not 0

MPI_Recv(&(A[0]), 1, partial_array, 1, 0, MPI_COMM_WORLD, &status);

source rank should be 0, not 1.

Otherwise you do not understand the MPI type matching correctly. It only states that underlying primitive types in the type maps on both ends should match. You are creating a vector whose type map has 20 primitive integers. If you send one element of this type, your message will actually contain 20 integers. On the receiver side you provide space for at least 20 integers so this is correct. The opposite is also correct.

It is not correct if you send only 10 or 18 integers in the second program since they will not make a complete element of the vector type. Nevertheless, the receive operation will complete but if you call MPI_Get_count() on the status, if will return MPI_UNDEFINED because from the number of received primitive integer elements one cannot construct an integer number of vector elements. It is also not correct to mix primitive types, e.g. send MPI_DOUBLE (or vector, or structure, or whatever other type that has doubles) and receive it as MPI_INT.

Please also note that MPI messages do not carry their type map or type ID with them so most MPI implementations do not check if types match. It is possible to send MPI_FLOAT and receive it as MPI_INT (because both are 4 bytes on most systems) but it is not correct to do so.