Question

So I have put together a few Python C-extensions and although their respective behaviors are verified, I’d like to verify this by some refcount debugging.

How can I verify that I have inserted the INC/DEC refcounts properly? I want to do this verification from a black box perspective, that is, how should a function’s input/output/memory look if all refcounts have been done properly?

My first approach would be to use sys.getrefcount(obj) and check all the input/output objects’ refcounts to see they are ok. Secondly, I could also perhaps check the memory used to see that there is no memory leak.

But I have never really done this before, so what’s the easiest and most correct way to do this?

Please note:
I don’t want to use any Cython style libraries, I’d rather do the C nitty gritty myself to learn the fundamentals.

Solution: So as suggested I built a debug enabled Python interpreter. I thought it would be tricky, but it wasn’t. As per this recipe I added those exact lines to Objects/object.c. After that all I had to do in the downloaded Python source directory was:

./configure --with-pydebug --prefix=/usr/local/python/
make
make install

With regards to building the C-extension to this new interpreter, all that had to be done on that front was to point the include_dirs tag in the Extension in setup.py to '/usr/local/python/include/python2.7', and then running the build and install command with the new interpreter. Once that was done I could add just add _Py_CountReferences(stderr); lines in the c code and the refcount would be dumped to stderr (console). Very good to see what changes the refcount, and how it increments/decrements as the code progresses.

Answer 1

One suggestion: Compile the Python interpreter in debug mode (–with-pydebug). Use this interpreter to compile and test your extension. The interpreter displays the count of the total number of objects and this total shouldn’t increase as you repeated run your tests. Make sure to test all the possible exit paths of your code – I’ve forgotten to DECREF objects when an exception occurred.