Question

I want to make sure I implement a barrier correctly. Right now, amStaticThreaded starts a ton of new threads every time it is called. In my main routine, everything after amStaticThreaded can be done by only one thread, but that part is extremely fast. Is there a better way than to have amStaticThreaded start a new set of threads every time

I have the following setup in my code:

void amStaticThreaded(int nshocks, int nstates,
                      MatrixXd& valmat, MatrixXd& EV_prev, MatrixXd& fullmat) {

  #pragma omp parallel for
  for(int i = 0; i < nshocks; i++) {

    // this does the add part    
    RowVectorXd vrow(nstates);
    vrow = EV_prev.row(i);

    fullmat.middleRows( i*nstates, nstates).rowwise() += vrow;
    valmat.row(i)   = fullmat.middleRows(i*nstates, nstates).rowwise().maxCoeff().transpose();

  }

}

int main() {

  // ...

  // ITERATION ON CONTINUATION VALUE  
  cout << "entering loop" << endl;
  while ( (err > TOL) && (itercount < MAXIT)) {

    // GET NEW EXPECTED VALUE FUNCTION
    EV_prev = (T_BIG * V_prev);
    EV_prev.array() *= beta;

    fullmat = staticmat;
    amStaticThreaded(Nshocks, nstates, V_new, EV_prev, fullmat);

    // THERE SHOULD BE A BARRIER HERE

    // FIGURE OUT THE ERROR BOUNDS
    mmdiff = (V_new.array() - V_prev.array());
    lbound = beta * double(mmdiff.minCoeff());
    ubound = beta * double(mmdiff.maxCoeff());

    // POSSIBLY ADJUST THE VALUE FUNCTION
    relres = ubound - lbound;
    if(relres < TOL) {
      V_new.array() += (ubound + lbound)/2.0;
    }
    err = relres;

    // UPDATE ITER COUNT AND VALUE FUNCTION
    cout << "i: " << itercount << ": " << err << endl;
    itercount++;
    V_prev = V_new;
  } // end while
  cout << "DONE with iteration!" << endl;
}

Answer 1

Your code is correct as it is. There is an implied barrier at the end of the parallel section; only the master thread executes instructions outside the parallel section.

As for the starting of many threads every time you enter the parallel for, this is something the OpenMP implementation will take care of. Most implementations keep a pool of threads and use them when needed, instead of always creating and deleting threads.