Question

I’m using a Tesla, and for the first time, I’m running low on CPU memory instead of GPU memory! Hence, I thought I could cut the size of my host memory by switching all integers to short (all my values are below 255).

However, I want my device memory to use integers, since the memory access is faster. So is there a way to copy my host memory (in short) to my device global memory (in int)? I guess this won’t work:

short *buf_h = new short[100];
int *buf_d = NULL;

cudaMalloc((void **)&buf_d, 100*sizeof(int));

cudaMemcpy( buf_d, buf_h, 100*sizeof(short), cudaMemcpyHostToDevice );

Any ideas? Thanks!

Answer 1

There isn’t really a way to do what you are asking directly. The CUDA API doesn’t support “smart copying” with padding or alignment, or “deep copying” of nested pointers, or anything like that. Memory transfers require linear host and device memory, and alignment must be the same between source and destination memory.

Having said that, one approach to circumvent this restriction would be to copy the host short data to an allocation of short2 on the device. Your device code can retrieve a short2 containing two packed shorts, extract the value it needs and then cast the value to int. This will give the code 32 bit memory transactions per thread, allowing for memory coalescing, and (if you are using Fermi GPUs) good L1 cache hit rates, because adjacent threads within a block would be reading the same 32 bit word. On non Fermi GPUs, you could probably use a shared memory scheme to efficiently retrieve all the values for a block using coalesced reads.