Question

I have developed a python C-extension that receives data from python and compute some cpu intensive calculations.
It’s possible to profile the C-extension?

The problem here is that writing a sample test in C to be profiled would be challenging because the code rely on particular inputs and data structures (generated by python control code).

Do you have any suggestions?

Answer 1

I’ve found my way using google-perftools. The trick was to wrap the functions StartProfiler and StopProfiler in python (throught cython in my case).

To profile the C extension is sufficient to wrap the python code inside the StartProfiler and StopProfiler calls.

from google_perftools_wrapped import StartProfiler, StopProfiler
import c_extension # extension to profile c_extension.so

StartProfiler("output.prof")
... calling the interesting functions from the C extension module ...
StopProfiler()

Then to analyze for example you can export in callgrind format and see the result in kcachegrind:

pprof --callgrind c_extension.so output.prof > output.callgrind 
kcachegrind output.callgrind