As there are multitude of ways to write binary modules for python, i was hopping those of you with experience could advice on the best approach if i wish to improve the performance of some segments of the code as much as possible.
As i understand, one can either write an extension using the python/numpy C-api, or wrap some already written pure C/C++/Fortran function to be called from the python code.
Naturally, tools like Cython are the easiest way to go, but i assume that writing the code by hand gives better control and provide better performance.
The question, and it may be to general, is which approach to use. Write a C or C++ extension? wrap external C/C++ functions or use callback to python functions?
I write this question after reading chapter 10 in Langtangen’s “Python scripting for computational science” where there is a comparison of several methods to interface between python and C.
I would say it depends on your skills/experience and your project.
If this is very ponctual and you are profficient in C/C++ and you have already written python wrapper, then write your own extension and interface it.
If you are going to work with Numpy on other project, then go for the Numpy C-API, it’s extensive and rather well documented but it is also quite a lot of documentation to process.
At least I had a lot of difficulty processing it, but then again I suck at C.
If you’re not really sure go Cython, far less time consuming and the performance are in most cases very good. (my choice)
From my point of view you need to be a good C coder to do better than Cython with the 2 previous implementation, and it will be much more complexe and time consuming.
So are you a great C coder ?
Also it might be worth your while to look into pycuda or some other GPGPU stuff if you’re looking for performance, depending on your hardware of course.