Question

This should be very simple but I could not find an exhaustive answer:

I need to perform A+B = C with matrices, where A and B are two matrices of unknown size (they could be 2×2 or 20.000×20.000 as greatest value)

Should I use CUBLAS with Sgemm function to calculate?

I need the maximum speed achievable so I thought of CUBLAS library which should be well-optimized

Answer 1

For any sort of technical computing, you should always use optimized libraries when available. Existing libraries, used by hundreds of other people, are going to be better tested and better optimized than anything you do yourself, and the time you don’t spend writing (and debugging, and optimizing) that function yourself can be better spent working on the actual high-level problem you want to solve instead of re-discovering things other people have already implemented. This is just basic specialization of labour stuff; focus on the compute problem you want to solve, and let people who spend their days professionally writing GPGPU matrix routines do that for you.

Only when you are sure that existing libraries don’t do what you need — maybe they solve too general a problem, or make certain assumptions that don’t hold in your case — should you roll your own.

I agree with the others that in this particular case, the operation is pretty straightforward and it’s feasible to DIY; but if you’re going to be doing anything else with those matricies once you’re done adding them, you’d be best off using optimized BLAS routines for whatever platform you’re on.