Question

Is it possible that cudaMalloc fails to allocate because there is no free computer memory?
Although the GPU memory is available cudaMalloc fails when the RAM (computer memory) is full.
My code can be sumamrized in the following example code:

int main()
{
    size_t N=sizeof(int)*100000000;
    while(true)
    {
        int *d_a,*d_b;
        if ( cudaSuccess !=cudaMalloc(&d_a, N)) printf("Error Allocating GPU Memory");
        if ( cudaSuccess !=cudaMalloc(&d_b, N)) printf("Error Allocating GPU Memory"); 
        cudaMemset(d_a,1,N);
        cudaMemset(d_b,2,N);

        int *h_a= (int *)malloc(N);
        int *h_b=(int *)malloc(N);
        if(!h_a || !h_b)  printf("Error Allocating CPU Memory");
        cudaMemcpy(d_a,h_a, N, cudaMemcpyHostToDevice);
        cudaMemcpy(d_b,h_b, N, cudaMemcpyHostToDevice);

        cudaFree(d_a);
        cudaFree(d_b);
    }
    getch();
    return 1;
}

The following code fails to allocate GPU memory only when computer memory is full!
The error i get is Error Allocating GPU Memory instead of Error Allocating CPU Memory

Answer 1

You’re freeing GPU memory in every iteration so that will never be full. Operating System uses your hardisk as Virtual Memory when the RAM gets full so the code will slow down but it will keep on allocating memory on Host side. Maybe CUDA keeps the pointer to the Device memory on Host memory which might be causing problem when RAM fills up.

An easy way to check this is to put a counter and see at which iteration the error comes. Then first run the program allocating memory on GPU alone and later on CPU alone, if the counter increases for both the cases then my assumption is correct.