Question

I am trying to implement Tournament barrier using MPI. Here, is the code I have written. I am writing only the arrival phase and wake up phase

//Arrival phase
while(1)
{
    if((!strcmp(round[my_id][round_num].role,"winner"))||(!strcmp(round[my_id][round_num].role,"champion")))
    {
       printf("%d is the winner of round %d\n",my_id,round_num);
       MPI_Recv(&reach_msg, sizeof(reach_msg), MPI_BYTE, round[my_id][round_num].opponent, tag, MPI_COMM_WORLD, &status);
       printf("%d received: %s\n",my_id,reach_msg);
    }

    else if(!strcmp(round[my_id][round_num].role,"loser"))
    {
       printf("%d is the loser of round %d\n",my_id,round_num);
       sprintf(reach_msg,"%d arrived at the barrier",my_id);
       MPI_Send(reach_msg,strlen(reach_msg+1),MPI_BYTE,round[my_id][round_num].opponent,tag,MPI_COMM_WORLD);
       MPI_Recv(wakeup_msg,sizeof(wakeup_msg),MPI_BYTE,round[my_id][round_num].opponent,tag,MPI_COMM_WORLD,&status);
       printf("%d received: %s\n",my_id,wakeup_msg);
    }

if(round_num==num_rounds)
       break;
    else
       round_num++;
}

printf("%d is out of arrival tree\n",my_id);

//wakeup tree
      while(1)
     {
       printf("%d prints: round num is: %d\n",my_id,round_num);
       if(round_num==0)
            break;
       sprintf(wakeup_msg,"wakeup msg from %d of %d",my_id,P);

       if((!strcmp(round[my_id][round_num].role,"winner"))||(!strcmp(round[my_id][round_num].role,"champion")))
       MPI_Send(wakeup_msg,strlen(wakeup_msg+1),MPI_BYTE,round[my_id][round_num].opponent,tag,MPI_COMM_WORLD);
       round_num--;
      }

    MPI_Finalize();
    return 0;
   }

I dont understand why race conditions occur. I believe the MPI_Send and MPI_Recv are blocking functions. But, sometimes it doesn’t behave that way

EDIT: Here is a sample output where race condition happens. As you can see, 0 received message(“1 arrived at the barrier”) from 1 even before 1 sent the message.

0 is the winner of round 1

0 received: 1 arrived at the barrie

0 is the winner of round 2

1 is the loser of round 1

1 sending reach msg

2 is the winner of round 1

2 received: 3 arrived at the barrie

2 is the loser of round 2

2 sending reach msg

3 is the loser of round 1

3 sending reach msg

0 received: 2 arrived at the barrie

0 sending wakeup msg

0 sending wakeup msg

1 received: wakeup msg from 0 with tag at round 1

2 received: wakeup msg from 0 with tag at round 2

2 sending wakeup msg

3 received: wakeup msg from 2 with tag at round 1

Answer 1

For debugging mpi programs, print statements are generally not a very good solution. As the print statments need to be send to the console from different nodes over the network and are cached before.

To debug it you should let each processor write to an own output file. Or let only one processor print statements, where you disable caching by either using std::cerr or somehow disable the caching of printf.