Question

I’m using boost::python to embed python, this is how I do it:

void runCode(){
    Py_Initialize();
    //boost::python code goes here and embedded python code runs
    Py_Finalize();
}

it runs nicely for the first time, but when it is run again, I get this error:

LookupError: unknown encoding: utf8

and code does not run as expected, any help is appreciated.

Answer 1

Since you didn’t get an expert answer, I’m offering my learning from working on a similar problem.
Python have issues with reinitialization support. This is unfortunate if you need to restart the interpreter due to some error, or want to run many independent interpreters.

One issue there is leaking resources and memory (quoting from the above link):

Bugs and caveats: Dynamically loaded extension modules loaded by
Python are not unloaded. Small amounts of memory allocated by the
Python interpreter may not be freed (if you find a leak, please report
it). Memory tied up in circular references between objects is not
freed. Some memory allocated by extension modules may not be freed.
Some extensions may not work properly if their initialization routine
is called more than once; this can happen if an application calls
Py_Initialize() and Py_Finalize() more than once.

Another issue is many modules don’t support this properly, as can be seen for example in this SO thread. I think this is the problem you’re facing.

It seems that most Python applications work-around this problem:

• by having the engine run in a dedicated process ;
• by using subinterpreters which represent distinct execution states (of a common interpreter)

If the second one works for you, go ahead with it.