Question

This c code crash with a segmentation fault if I compile with GCC using -fomit-frame-pointer and -mrtd.

Is my code wrong in some way? Other function pointers works as expected, but not when its the free-function that is being passed? I get a warning when compiling, but I dont understand why or what I should do to fix. (I usually dont code in c, this error comes from a 3d part library that I use)

I need rtd/stdcall because Im on windows and need to call this library from python with ctypes, and -fomit-frame-pointer is included by default when compiling with GCC with -O1. (GCC version is 4.6.1 from TDM/Mingw32) It feels a bit strange that a default optimization option would cause trouble?

c-code:

#include <stdlib.h>

// void free ( void * ptr );
void test(void* o, void (*freeFunc)(void*)) {
    freeFunc(o);
}

int main() {
    int *p = (int *)calloc(1, sizeof(int));
    test(p, free);
}

compiled with:

gcc -fomit-frame-pointer -mrtd -c fx.c
gcc -fomit-frame-pointer -mrtd fx.o -o fx.exe

compile warning:

fx.c: In function 'main':
fx.c:11:5: warning: passing argument 2 of 'test' from incompatible pointer type[enabled by default]
fx.c:5:6: note: expected 'void (*)(void *)' but argument is of type 'void (*)(void *)'

Answer 1

From the GCC manual page for -mrtd:

Warning: this calling convention is incompatible with the one
normally used on Unix, so you cannot use it if you need to call
libraries compiled with the Unix compiler.

The warning is kind of bizarre, but I believe it’s simply trying to tell you that you’re passing a pointer to a function which uses an incompatible calling convention. I imagine the free in libc would be such a function; I’m surprised that calling calloc works at all. (Or perhaps it isn’t, and what you’re seeing is a delayed failure!)