I have a weird problem with defining a mpi_type_contiguous and using mpi_gatherv later on.
The type is defined as:
type glist
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!uncomment line below for int version:
! integer :: iref , biref
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
real(8) :: rvar
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!uncomment line below for buggy version:
integer :: ciref
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
end type glist
The code as it is now doesn’t work. If I would comment integer :: ciref it would work. The same is true if I comment real(8) :: rvar instead and uncomment the two other integers integer :: iref, biref.
That means that the mistake depends both on the size of the datatype but only if there is a real(8) in there. If I have one real(8) and two int then it works again.
The code is designed to run with 3 threads(!). I was running it with openmpi and gfortran (mpif90). No special compile flags and execution with mpirun -np 3 filename. If somebody could run it with mpich or compile it with ifort or whatever that would be interesting so as to find out where the problem is coming from.
— EDIT —
Platinummonkey suggested below to use mpi_type_struct but it still doesn’t work. If I do a sizeof(glist) with glist as above, I get 16 as an answer instead of 12.
— /EDIT —
Thanks in advance for your help.
The full code is (don’t worry some of it can be ignored)
module mod_glist
type glist
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!uncomment line below for int version:
! integer :: iref , biref
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
real(8) :: rvar
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!uncomment line below for buggy version:
integer :: ciref
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
end type glist
contains
subroutine sof_glist(sof)
implicit none
integer, intent(out) :: sof
type(glist) :: dum
integer :: val
val = 0
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!uncomment line below for int version:
! val = kind(dum%iref) + kind(dum%biref)
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
val = val + kind(dum%rvar)
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!uncomment line below for buggy version:
val = val + kind(dum%ciref)
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
sof = val/kind(0)
write(*,*) 'Size in bytes, integers: ', sof, val
end subroutine
end module mod_glist
program test_mpi_gatherv
use mpi
use mod_glist
implicit none
integer :: err, np, tp, nglout, i, j, nglin, sofgl, mpi_type_glist
type(glist), dimension(:), allocatable :: gl, glcom, glsave
integer , dimension(:), allocatable :: glsize, nglinv, nglinp
integer(kind=mpi_address_kind) :: ii, ij
call mpi_init(err)
call mpi_comm_size(mpi_comm_world, np, err)
call mpi_comm_rank(mpi_comm_world, tp, err)
tp = tp + 1
call sof_glist(sofgl)
call mpi_type_contiguous(sofgl, mpi_integer, mpi_type_glist, err)
call mpi_type_commit(mpi_type_glist, err)
call mpi_type_get_extent(mpi_type_glist, ii, ij, err)
write(*,*) 'extend: ', ii, ij
allocate(glsize(np), nglinv(np), nglinp(np))
glsize(1) = 5
glsize(2) = 4
glsize(3) = 3
glsize(4:np) = 0
allocate(gl(glsize(tp)))
j = 1
do i = 1,tp-1
j = j+glsize(i)
enddo
do i = 1,glsize(tp)
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!uncomment line below for int version:
! gl(i)%iref = j
! gl(i)%biref = -j
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
gl(i)%rvar = real(j,8)
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
!uncomment line below for buggy version:
gl(i)%ciref = -j*10
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
j = j+1
enddo
do i=1,np ! setting up stuff can be ignored
if(i.eq.1)then
if(tp.eq.i)then
nglinv(1) = 0
nglinv(2) = 2
nglinv(3) = 3
nglinp(1) = 0
nglinp(2) = nglinv(1) + nglinp(1)
nglinp(3) = nglinv(2) + nglinp(2)
nglin = nglinv(1) + nglinv(2) + nglinv(3)
allocate(glcom(nglin))
nglout = 0
else
if(tp.eq.2)then
nglout = 2
allocate(glcom(nglout))
glcom(1) = gl(1)
glcom(2) = gl(3)
elseif(tp.eq.3)then
nglout = 3
allocate(glcom(nglout))
glcom(1) = gl(1)
glcom(2) = gl(2)
glcom(3) = gl(3)
endif
endif
elseif(i.eq.2)then
if(tp.eq.i)then
nglinv(1) = 3
nglinv(2) = 0
nglinv(3) = 2
nglinp(1) = 0
nglinp(2) = nglinv(1) + nglinp(1)
nglinp(3) = nglinv(2) + nglinp(2)
nglin = nglinv(1) + nglinv(2) + nglinv(3)
allocate(glcom(nglin))
nglout = 0
else
if(tp.eq.1)then
nglout = 3
allocate(glcom(nglout))
glcom(1) = gl(2)
glcom(2) = gl(4)
glcom(3) = gl(5)
elseif(tp.eq.3)then
nglout = 2
allocate(glcom(nglout))
glcom(1) = gl(2)
glcom(2) = gl(3)
endif
endif
elseif(i.eq.3)then
if(tp.eq.i)then
nglinv(1) = 0
nglinv(2) = 2
nglinv(3) = 0
nglinp(1) = 0
nglinp(2) = nglinv(1) + nglinp(1)
nglinp(3) = nglinv(2) + nglinp(2)
nglin = nglinv(1) + nglinv(2) + nglinv(3)
allocate(glcom(nglin))
nglout = 0
else
if(tp.eq.1)then
nglout = 0
allocate(glcom(nglout))
elseif(tp.eq.2)then
nglout = 2
allocate(glcom(nglout))
glcom(1) = gl(1)
glcom(2) = gl(4)
endif
endif
endif ! end of setting up stuff
if(i.eq.tp) allocate(glsave(nglin))
! debug output
call mpi_barrier(mpi_comm_world, err)
write(*,*) i, tp, nglout, nglin
call mpi_barrier(mpi_comm_world, err)
if(i.eq.tp) write(*,*) i, nglinv, nglinp
call mpi_barrier(mpi_comm_world, err)
! end debug output
call mpi_gatherv(glcom, nglout, mpi_type_glist, glsave, nglinv, nglinp, mpi_type_glist, i-1, mpi_comm_world, err)
if(allocated(glcom)) deallocate(glcom)
enddo
! debug output
call mpi_barrier(mpi_comm_world, err)
do i = 1,nglin
write(*,*) tp, i, glsave(i)
enddo
! end debug output
call mpi_finalize(err)
end program
Your basic error is that you cannot calculate the size of a derived type by summing the size of its components, because that ignores the padding which is necessary to satisfy alignment requirements. In your example, a real(8) needs to be aligned on a 8 byte boundary, so if the derived type contains a default kind integer (size 4 bytes), then the compiler will add 4 bytes of padding to ensure that the next element in the array of derived types will start on an 8 byte boundary. As pointed out in the answer by platinummonkey, the correct solution to this problem is to define an mpi_type_struct: MPI struct datatype with an array
Also, assuming that the kind numbers are equivalent to the size of a type is not portable, it just happens to work in gfortran.