Question

I’ve write a program which compute a calculation in parallel with openMP on data from on a file,

let says :

./foobar input.txt

I’m on the way to modify my program such a way that, It’s will do the same calculation but upon many files :

./foobar input1.txt input2.txt input3.txt

My question is :

what is suppose to be the more efficient between :
ready each file (which can reach hundred MegaByte in size) and do a calculation in parallel on each of them,

for (i = O; i < numberOfFile; i++)
  calculationOn(filename[i]); // the calculation program run in parallel

or let each thread read in parallel it own file and work on it ?

#pragma omp parallel for private(i)
for(i = 0; i < numberOfFile; i++)
  calculationOn(filename[i]);

thanks for any reply !

Answer 1

If you have very many files and the output per file is independent of all the other files, then you don’t need OpenMP at all. Just run entire the program in parallel on multiple processors with a tool like GNU Parallel to get linear speedup. An OpenMP loop over the arguments is likely quite wasteful in these cases; in terms of programmer time, that is.