Question

I am a beginner at MPI and I would like to use threads with MPI in the following way. Certain processes (all even procs) spawn threads and do a blocking wait to receive a message. The number of threads spawned per proc is a command line argument. So if there are 4 procs, processes 0 and 2 spawn a thread. Now, I want both process 0 and 2 to send message to all the threads. For instance, process 0 sends message to itself AND to process 2, and proc 2 sends it to proc 0 and itself.
Here is what my code looks like, and it apparently does not do what is desired. It just waits to receive message. Where am I going wrong?

Thanks!

  typedef struct {
     int id;
  } struct_t;

void *hello(void *arg)
{
    int rank;
    char mystr[10];
    MPI_Status status;
    struct_t *fd=(struct_t *)arg;
    MPI_Comm_rank(MPI_COMM_WORLD, &rank);
    printf("Rank %d is waiting in thread %d for my message\n", rank, fd->id);

    if(rank%2 ==0){
            MPI_Recv(mystr, 10, MPI_CHAR, MPI_ANY_SOURCE, 0, MPI_COMM_WORLD, &status);
            printf("Thread %d on rank %d received %s\n", fd->id, rank, mystr);
    }

    return (NULL);
}

void spawn_thread(int n)
{
    int size,rank, i;
    pthread_t *threads;
    pthread_attr_t pthread_custom_attr;
    struct_t *fd;
    threads=(pthread_t *)malloc(n*sizeof(*threads));
    pthread_attr_init(&pthread_custom_attr);
    fd=(struct_t *)malloc(sizeof(struct_t)*n);

    MPI_Comm_rank(MPI_COMM_WORLD, &rank);
    MPI_Comm_size(MPI_COMM_WORLD, &size);

    /* Start up thread */
    for (i=0; i<n; i++)
    {
            fd[i].id=i;
     //       printf("My rank is %d and I created thread #%d\n", rank, i);
            pthread_create(&threads[i], &pthread_custom_attr, hello, (void *)(fd+i));
    }

    /* Synchronize the completion of each thread. */
    for (i=0; i<n; i++)
    {
            pthread_join(threads[i],NULL);
    }
    free(fd);
}
 void main(int argc, char ** argv)
{
    int n,i, provided, claimed;
    int rank, size, errs;

    MPI_Init_thread(&argc, &argv, MPI_THREAD_MULTIPLE, &provided);
    MPI_Comm_rank(MPI_COMM_WORLD, &rank);
    MPI_Comm_size(MPI_COMM_WORLD, &size);

    if (argc != 2)
    {
            printf ("Usage: %s n\n  where n is no. of threads\n",argv[0]);
            exit(1);
    }

    n=atoi(argv[1]);
    if ((n < 1) || (n > MAX_THREAD))
    {
            printf ("The no of thread should between 1 and %d.\n",MAX_THREAD);
            MPI_Abort(MPI_COMM_WORLD,-1);
    }

  if(rank%2 == 0)
            spawn_thread(n);

    if(rank%2 == 0){

                    printf("My rank is %d and I am sending Hello!\n", rank);
                    MPI_Send("HELLOOO", 10, MPI_CHAR, rank, 0, MPI_COMM_WORLD);
          }

    MPI_Finalize();
}

Answer 1

I’m not entirely sure I understand what you are trying to achieve, but please not that all your even ranked processes’ threads will block on the receive, so nobody is going to run the send code. Your odd ranked processes’ threads just start and finish immediately as they won’t do anything.

Maybe the following if:

    if(rank%2 == 0){
      printf("My rank is %d and I am sending Hello!\n", rank);
      MPI_Send("HELLOOO", 10, MPI_CHAR, rank, 0, MPI_COMM_WORLD);
    }

should be like this:

    if(rank%2 != 0)

This way your odd ranked processes’ would at least send the command?

Alternatively, you need to move the ‘join’ code outside the spawn_thread function and do the join after the send is called.

Hope this helps.