Question

I need to write code to do some realtime processing that is fairly computationally complex. I would like to create some Python classes to manage all my scripting, and leave the intensive parts of the algorithm coded in C++ so that they can run as fast as possible. I would like to instantiate the objects in Python, and have the C++ algorithms chime back into the script with callbacks in python. Something like:

myObject = MyObject()
myObject.setCallback(myCallback)
myObject.run()

def myCallback(val):
    """Do something with the value passed back to the python script."""
    pass

Will this be possible? How can I run a callback in python from a loop that is running in a C++ module? Anyone have a link or a tutorial to help me do this correctly?

Answer 1

I suggest using Boost.Python as suggested by ChristopheD. A gotcha would be if the C++ extension is running in it’s own thread context (not created by Python). If that’s the case, make sure to use the PyGILState_Ensure() and PyGILState_Release() functions when calling into Python code from C++.

From the docs (http://docs.python.org/c-api/init.html#thread-state-and-the-global-interpreter-lock):

Beginning with version 2.3, threads
can now take advantage of the
PyGILState_*() functions to do all of
the above automatically. The typical
idiom for calling into Python from a C
thread is now:

PyGILState_STATE gstate;
gstate = PyGILState_Ensure();

/* Perform Python actions here.  */
result = CallSomeFunction();
/* evaluate result */

/* Release the thread. No Python API allowed beyond this point. */
PyGILState_Release(gstate)

I recommend making the callbacks short & sweet – to limit the need to perform exception handling in C++ code. If you’re using wxPython, you could use it’s robust async event system. Or the callbacks could put events on a Queue and you could have a thread devoted to asynchronously executing callback/event code.

Even with Boost.Python magic, you’ll have to get familiar with this portion of the Python C API when dealing with threads. (Don’t forget to wrap the C++ functions with Py_BEGIN_ALLOW_THREADS and Py_END_ALLOW_THREADS to release the GIL!)