Question

I’m running an algorithm at the moment that is very heavy but extremely parallel.

I’ve been looking at ways to speed it up and I’ve noticed that the slowest operation I have is my VecAdd function (It gets called thousands of times on a 6000 or so wide vector).

It is implemented as follows:

bool VecAdd( float* pOut, const float* pIn1, const float* pIn2, unsigned int num )
{
    for( int idx = 0; idx < num; idx++ )
    {
        pOut[idx]   = pIn1[idx] + pIn2[idx];
    }
    return true;
}

Its a very simple loop but all the additions can be performed in parallel. My first optimisation option is to move over to using SIMD as I can easily get a near 4 times speed up doing this.

However I’m also interested in the possibility of using OpenMP and having it automatically thread the for loop (potentially giving me a further 4x speedup for a total of 16x with SIMD).

However it really runs slowly. With the loop straight it takes around 3.2 seconds to process my example data. If I insert

#pragma omp parallel for

before the for loop I was assuming it would farm out several blocks of additions to other threads.

Unfortunately the result is that it takes ~7 seconds to process my example data.

Now I understand that a lot of my problem here will be caused by overheads with setting up threads and so forth but I’m still surprised just how much slower it makes things run.

Is it possible to speed this up by somehow setting up the thread pool in advance or will I never be able to combat these overheads?

Any thoughts on advice as to whether I can thread this nicely with OpenMP will be much appreciated!

Answer 1

Your loop should parallelize fine with the #pragma omp parallel for.
However, I think the problem is that you shouldn’t parallelize at that level. You said that the function gets called thousands of times, but only operates on 6000 floats. Parallelize at the higher level, so that each thread is responsible for thounsands/4 calls to VecAdd. Right now you have this algorithm:

1. List item
2. serial execution
3. (re) start threads
4. do short computation
5. synchronize threads (at the end of the for loop)
6. back to serial code

Change it so that it’s parallel at the highest possible level.

Memory bandwidth of course matters, but there is no way it would result in slower than serial execution.