Question

I have a weird problem, so I thought I would ask and see if someone more experienced than me could see a solution.

I am writing a program with CUDA C/C++, and I have some constant integers that specify various things, like coordinates of the bounds of the calculation, etc.. Currently I just have those things in global device memory. They are accessed by every thread in every kernel call, and so I figured that if they are in global memory, then they never are being cached or broadcast (right?). And so these little integers are taking up a lot (relatively) of overhead, and have a lot of ‘read redundancy.’

So I declare in a header:

__constant__ int* number;

I include that header, and, when I do memory stuff, I do:

cutilSafeCall( cudaMemcpyToSymbol(number, &(some_host_int), sizeof(int) );

I pass number into all my kernel’s then:

__global__ void magical_kernel(int* number, ...){

   //and I access 'number' like this
   int data_thingy = big_array[ *number ];

}

My code crashes. With number in global memory, it is just fine. I have determined that it crashes sometime upon accessing number within the kernel. This means that either I am accessing or allocating it wrong. If it holds the wrong value, it will also cause a crash, because it is used to index into arrays.

To conclude, I will ask a few questions. First, what am I doing wrong? As a bonus: is there a better way than constant memory to accomplish this task – I don’t know the value of number at compile time, so a simple #define won’t work. Will constant memory even speed the code up at all, or has it been cached and broadcasted all along? Could I somehow put the data in shared memory for each threadblock and have it remain in shared memory through multiple kernel calls?

Answer 1

There are several problems here:

1. You have declared number as a pointer, but never assigned it a value which is valid address in GPU memory
2. You have a variable scope onflict: the argument variable int * number defined in magic_kernel is not the same variable as the __constant__ int * variable defined as compilation unit scope.
3. The first argument of the cudaMemcpyToSymbol call is almost certainly incorrect.

If you don’t understand why either of the first two point are true, you have some revision to do on pointers and scope in C++.

Based on your response to a now deleted answer, I suspect what you are actually trying to do is this:

__constant__ int number;

__global__ void magical_kernel(...){

   int data_thingy = big_array[ number ];

}

cudaMemcpyToSymbol("number", &(some_host_int), sizeof(int));

i.e. number is intended to be an integer in constant memory, not a pointer, and not a kernel argument.

---

EDIT: here is an exmaple which shows this in action:

#include <cstdio>

__constant__ int number;

__global__ void magical_kernel(int * out)
{
   out[threadIdx.x] = number;
}

int main()
{
    const int value = 314159;
    const size_t sz = size_t(32) * sizeof(int);
    cudaMemcpyToSymbol("number", &value, sizeof(int));

    int * _out, * out;

    out = (int *)malloc(sz);
    cudaMalloc((void **)&_out, sz);

    magical_kernel<<<1,32>>>(_out);

    cudaMemcpy(out, _out, sz, cudaMemcpyDeviceToHost);
    for(int i=0; i<32; i++)
        fprintf(stdout, "%d %d\n", i, out[i]);

    return 0;
}

You should be able to run this yourself and confirm it works as advertised.