Question

I am working on CUDA and I have a problem related to thread synchronization. In my code I need threads to execute different parts of the code, like:

one thread -> 
all thread ->
one thread ->

This is what I want. In the initial part of code only one thread will execute and then some part will be executed by all threads then again single thread. Also the threads are executing in a loop. Can anyone tell me how to do that?

Answer 1

You can only synchronize threads within a single blocks. It is possible to synchronize between multiple blocks, but only under very specific circumstances. If you need global synchronization between all threads, the way to do that is to launch a new kernel.

Within a block, you can synchronize threads using __syncthreads(). For example:

__global__ void F(float *A, int N)
{
    int idx = threadIdx.x + blockIdx.x * blockDim.x;

    if (threadIdx.x == 0) // thread 0 of each block does this:
    {
         // Whatever
    }
    __syncthreads();

    if (idx < N) // prevent buffer overruns
    {
        A[idx] = A[idx] * A[idx];  // "real work"
    }

    __syncthreads();

    if (threadIdx.x == 0) // thread 0 of each block does this:
    {
         // Whatever
    }
}