Question

i am calling a nested for loop as follows:

do ir = 1,Nr
    do iom = iom1, iom2
       xyz(1) = xo(1) + xom(iom)*r
       xyz(2) = xo(2) + yom(iom)*r
       xyz(3) = xo(3) + zom(iom)*r
       call FUNSUB(xyz,Nprop,PropXYZ)
    enddo
enddo

where FUNSUB evaluates a property in the following manner:

id = 1
do l=0,lmax
    do m=-l,l
        id = id + 1
        Prop(id) = RHO * Ylm(l,m,xl(1),xl(2),xl(3))
    enddo
enddo

now i am trying to parallelize this with something of the form

!$OMP parallel do reduction(+:Prop) private(ir, l, m, j, iom, r, wrad, xyz, PropOm, PropOm1, PropXYZ)

where some of the other private variables seen here are left out of my example code for brevity. i am intentionally trying to make l and m private so that inside the FUNSUB loops, the properties are evaluated independently. i find, however, that they are shared between threads. i have put in some debugging options and found that id continually goes above the bounds of Prop and have found that l also goes about lmax as well as m being outside the bounds of (-l,l).

my question is: how do i go about making sure that l and m are kept private and NOT shared from the main routine? it seems that the problem is that it is an entirely different subroutine which holds these values and that, somehow these private declarations of variables do not carry over

Answer 1

I think your problem is that in Fortran, local subroutine variables can be static and that the !$OMP does not propagate to called functions.

Your code should work as expected if you copy-paste FUNSUB into the calling loops, i.e. if you manually inline it.

Update

After reading-up a bit on the looniness of how local variables may or may not be implemented by the compiler, I guess your best option is to declare FUNSUB as RECURSIVE. This forces all local variables onto the stack, meaning that each OpenMP thread which calls FUNSUB will have its own, private, set of local variables.