Question

Below is a small program that i wrote to see how race conditions can happen in CUDA but i was surprised by the output.

#include<cutil.h>
#include<iostream>
__global__ void testLocal(int *something, int val[]){

 *something = *something/2;


 val[threadIdx.x] = *something;
}

void main(){

    int *a, *c;
    int r =16;

    cudaMalloc((void**)&a, 4*sizeof(int));
    cudaMalloc((void**)&c, sizeof(int));
    cudaMemcpy(c, &r, sizeof(int) , cudaMemcpyHostToDevice);
    testLocal<<<1,4>>>(c,a);
    int *b = (int *)malloc(4 * sizeof(int));
    cudaMemcpy(b,a, 4 * sizeof(int), cudaMemcpyDeviceToHost);

    for( int  j =0 ; j< 4; j++){
        printf("%d\n",b[j]);

    }
    getchar();


}

As i am launching 4 threads, I expected each thread to divide *something by 2 once. I understand that the order in which they would divide *something is not fixed. Thus, when I tried to print the values, I expected that one of the printed values would be 8 , one would be 4, one would be 2, and one would be 1. However, all the printed values were 8. Why is this? Shouldn’t all threads divide *something once.

Answer 1

What you are looking at is undefined behaviour. Because you are launching a single block with 4 threads, all the threads are executing in the same warp. This means that

 *something = *something/2;

is being executed simultaneously by all the threads you have launched. The CUDA programming model only guarantees that when multiple threads from the same warp attempt to write to the same memory location, one of the writes will succeed. It says nothing about which thread will succeed, and what will happen to the other threads in the warp that don’t “win”. To get the behaviour you are expecting would require serialised memory access — this is only possible through the use of atomic memory access primitives on those architectures which support them.