Question

Using MPI, how do you wait for threads to finish?

For example:

for (int i = localstart; i < localend; i++)
{
    // do stuff that is computationally intensive
}
// I need to wait for all other threads to finish here
if (rank == 0) do_something();

Answer 1

If by threads you meant processes/ranks, then the answer is MPI_Barrier.

But look at the other collective operations too: they might make sense in your application, and offer better performance than hand-coding communication. For example, you could use MPI_Allgather to communicate all data to all ranks, and so on.

If you meant threads (like pthreads), then you’d have to use whatever the threading library offers.