I am attempting to call a C++ API from python. Following is the code in the pseudo form.
class Engine { // Singleton Class which does a heavy duty work
public:
static Engine* getEngine();
bool init();
private:
static Engine* m_instance;
Engine();
};
// Following the code to wrap the call to engine to call from python
// Its only a simplified form
//engine_module.c
#include <Engine.h>
PyObject* initengine() {
Engine* e = Engine::getInstance();
e->init();
// return the Py_BuildValue ...
}
PyObject* initengine_module() {
//... init the module
}
// Python code
import engine_module
status = engine_module.init()
Problem:
The Engine class is in libengine.so and when it inits it fails because internally dynamic_cast fails. The Engine in turns loads other libraries using dlopen(). I searched the net to add RTDL_GLOBAL and -E option while linking but still its not resolved. Am I supposed to add the -E option while compiling the python itself?
What could be the reason that Engine class works perfectly well when used in C++ code and does not work when used in python?
Edit 1:
To clarify on the question from Cat++:
libengine.so has many other classes which internally in Engine::init() uses dynamic_cast<>. The classes involved in dynamic_cast are not exposed to python at all. Only the Engine::init() is exposed.
Edit 2:
The platform is Red Hat Linux and the compiler is Intel
The question is how and when the dynamic library are loaded. The code
you show in engine_module.c references
Engine, so the library withEnginewill be automatically loaded before any of the initializationcode in
engine_module.cis executed. Similarly, any libraries used byEnginewill be loaded before the library withEngineis loaded. Allof which will be loaded using the flags Python used to load its
interface module. (
RTDL_LOCALis my guess.) Anydlopenyou invokelater will find that the module has already been loaded, and ignore the
request—including ignoring any options to
dlopenyou might havepassed.
The way we solved this was to create a special loader module, which
contained no direct references to any of the other modules. Python
loads this module, which implements the
initxxxfunction. Theinitxxxfunction explicitly loads all of the other modules needed, ina dependency determined order. (If A used B, B will be loaded before A.)
With
RTDL_GLOBAL, of course. In your case, this would be thelibraries used by
Engine, thenEngine. The last module to be loadedwould be the one with the Python interface; we put the Python
initialization code in the constructor of a static object, so it would
be executed automatically when the object was loaded, but you can also
put it in a named function, as long as you get the address of this
function using
dlsym, rather than declaring itexternin anyfashion. The important thing is to ensure that all of the libraries are first loaded by your explicit
dlopen, and not implicitly as a result of an undefined external in some library loaded earlier.