Question

I am teaching myself MPI. I am reading Python documentation on reduce and scan:
http://documen.tician.de/boostmpi/reference.html#boostmpi.scan

Both reduce and scan seem to take some function (op) and use it to reduce values obtained from individual processes into a single value.

How do reduce and scan differ?

Answer 1

A reduction means all processors get the same value while scan returns the partial operation results on each processor. For instance, if you had 10 processors and you were taking the sum of their rank, MPI_Reduce would give you the scalar 45 (0+1+2+3+4+5+6+7+8+9) on the root process only while MPI_scan would give you the scalar of the reduction up to the rank of the processor on each processor. So processor 0 would get 0, processor 1 would get 1, processor 2 would get 3, and so on. Processor 9 would get 45.

In other words, if you were to make a list of all processors values found from MPI_Scan it would be:

[0, 1, 3, 6, 10, 15, 21, 28, 36, 45]

with the scan result coming from [0, 0+1, 0+1+2, 0+1+2+3, ..., 0+1+2+3+4+5+6+7+8+9]

In Python, the list of results from MPI_Reduce would be (assuming processor 0 is the root):

[45, None, None, None, None, None, None, None, None, None]

while other languages the recvbuf would have undefined data in it on all processors other than the root.