A seemingly silly question but I can’t seem to find a definitive answer one way or the other.
The basic questions is do I need to have a corresponding MPI::Irecv for an MPI::Isend?
That is, even though the message sending is non-blocking, as long as I wait on the sends to complete before reusing the send buffers, do I need to use non-blocking receives & waits to receive the sent buffers?
My point is, I want to use non-blocking sends to “do other stuff” while the message is being sent but the receiver process will use the buffers immediately so I want them to block until the buffer is truly received.
It seems like I should be able to receive messages with MPI::Recv even though they were sent with MPI::Isend but I am wondering if I am missing something?
A bit of simple pseudo code
if( rank == 0 ){
int r;
for ( int i = 0; i < n; i++ ){
// DO SOME STUFF HERE...
request.Wait(status);
request2.Wait(status);
request3.Wait(status);
r = i;
memcpy( key, fromKey(i), ...);
memcpy( trace, fromTrace(i), ...);
request = MPI::COMM_WORLD.Isend( &r, 1, MPI::INT, node, tag );
request2 = MPI::COMM_WORLD.Isend( key, 10, MPI::INT, node, tag );
request3 = MPI::COMM_WORLD.Isend( trace, nBytesTotal, MPI::BYTE, node, tag );
// DO SOME MORE STUFF HERE.
}
r = -1;
request = MPI::COMM_WORLD.Isend( &r, 1, MPI::INT, node, tag );
// Carry on ...
} else {
int r = -1;
MPI::COMM_WORLD.Recv( &r, 1, MPI::INT, 0, tag, status );
while( r >= 0 ){
MPI::COMM_WORLD.Recv( &key, 10, MPI::INT, 0, tag, status );
memcpy( saveKey, key, ...);
MPI::COMM_WORLD.Recv( &trace, nBytesTotal, MPI::BYTE, 0, tag, status );
memcpy( saveTrace, trace, ...);
MPI::COMM_WORLD.Recv( &r, 1, MPI::INT, 0, tag, status );
}
No, you can freely mix blocking and non-blocking MPI operations on both ends of the communication. Blocking is related to when the MPI call returns control to your code and not to the content of the message(s) being transmitted.
Every MPI message carries an “envelope” with itself, that contains its source, destination, tag, and communicator. To successfully receive a message your receive operation should only match its envelope. The envelope in no way specifies how exactly was the message sent – was it via a blocking, was it via a non-blocking operation, was it a synchronous send (
MPI_Ssend) or a buffered one (MPI_Bsend). The only exception is the so-called “ready mode” send that is initiated withMPI_Rsend()orMPI_Irsend()which requires that the matching receive operation has already been posted or the message will not be delivered.That’s why the term “matching receive operation” is used throughout the MPI standard and not something like “corresponding receive function”.