Question

I would like to compare the performance of a serial program running on a CPU and a CUDA program running on a GPU. But I’m not sure how to compare the performance fairly. For example, if I compare the performance of an old CPU with a new GPU, then I will have immense speedup.

Another question: How can I compare my CUDA program with another CUDA program reported in a paper (both run on different GPUs and I cannot access the source code).

Answer 1

For fairness, you should include the data transfer times to get the data into and out of the GPU. It’s not hard to write a blazing fast CUDA function. The real trick is in figuring out how to keep it fed, or how to hide the cost of data transfer by overlapping it with other necessary work. Unless your routine is 100% compute-bound, including data transfer in your units-of-work-done-per-unit-of-time is critical to understanding how your implementation would handle, say, a lot more units of work.

For cross-device comparisons, it might be useful to report units of work performed per unit of time per processor core. The per processor core will help normalize large differences between, say, a 200 core and a 2000 core CUDA device.

If you’re talking about your algorithm (not just output), it is useful to describe how you broke the problem down for parallel execution – your block/thread distribution, for example.

Make sure you are not measuring performance on a debug build, or running in a debugger. Debugging adds overhead.

Make sure that your work sample is large enough that it is significantly above the “noise floor”. A test run that takes a few seconds to complete will be measuring more of your function and less of the ambient noise of the environment than a test run that completes in milliseconds. You can always divide the units of work by the test execution time to arrive at a sexy “units per nanosecond” figure, but you don’t actually measure it that way.