Question

In a C extension to my Python program, I am trying to improve performance by setting two of the inputs to the main C function (which is called millions of times) as global variables, as they don’t change often, so that I don’t have to keep supplying them from Python using a lambda wrapper to the C function (which wastes a nontrivial amount of time). My code looks like this. First I declare the global variables at the top of the file:

unsigned char* splitArray;
PyObject* wordCmp;

Then the Python API function to set them:

static PyObject* py_set_globals(PyObject *self, PyObject *args)
{
    free(wordCmp);
    free(splitArray);

    char* splitChars;
    PyObject* wordC;

    if (!PyArg_ParseTuple(args, "sO", &splitChars, &wordC))
        return NULL;

    wordCmp = (PyObject*)malloc(sizeof(PyObject));
    memcpy(wordCmp, wordC, sizeof(PyObject));
    splitArray = splitchars_to_mapping(splitChars);

    return Py_BuildValue("");
}

In this case, splitArray is assigned to a 128-character array which is malloc’ed in the function splitchars_to_mapping, and wordCmp is a Python function object that is passed to C. Anyway, as far as I can tell, the char* splitArray works fine as a global variable, but when I later try to call wordCmp using PyEval_CallObject, Python/C crashes. So I have two questions:

1. Why doesn’t C crash immediately when I try to free the uninitialized pointers wordCmp and splitArray at the beginning of the function?
2. Why am I unable to call wordCmp later when I have properly stored it on the heap and saved a reference to it as a global?

Answer 1

As to the first question, why it doesn’t crash when free’ing uninitialized global variables, that’s because global (and static) variables are initialized to zero when the program is loaded (guaranteed by the standard) and when you call free() with NULL (or zero) it does nothing.

From man free(1):

free(ptr) has already been called before, undefined behavior occurs.
If ptr is NULL, no operation is performed.

Edit:
Your second question is related to fact that you’re trying to copy a PyObject and you shouldn’t do that, as the PyObject structure could contain pointers and you can’t do deep copying because you don’t have access to that structure, you should however increment the reference count and keep the reference around for later use, note that when you use O the reference count is not incremented, from the docs:

O (object) [PyObject *]
Store a Python object (without any conversion) in a C object pointer. The C program thus receives the actual object that was
passed. The object’s reference count is not increased.

So you should increment the reference yourself, see this example

static PyObject *my_callback = NULL;

static PyObject* my_set_callback(PyObject *dummy, PyObject *args)
{
    PyObject *result = NULL;    
    if (PyArg_ParseTuple(args, "O:set_callback", &my_callback)) {
        if (!PyCallable_Check(my_callback)) {
            PyErr_SetString(PyExc_TypeError, "parameter must be callable");
            return NULL;
        }
        Py_XINCREF(my_callback); /* Add a reference to new callback */           
        Py_INCREF(Py_None);      /* Boilerplate to return "None" */
        result = Py_None;
    }
    return result;
}