Question

I’m trying to make a call to a DLL function (via GetProcAddress etc) from C, using lcc compiler. The function gets called and everything goes well, but it looks like the top of the stack gets corrupted. I’ve tried to play with calling conventions (__stdcall / __cdecl), but that didn’t help.

Unfortunately I don’t have access to the dll code, and have to use the lcc compiler.

I found that this simple hack avoids stack corruption:

void foo(params)
{
  int dummy;
  dll_foo(params);
}

Here dll_foo is the pointer returned by GetProcAddress, and the stack is kind of protected by the dummy variable. So it’s not the stack pointer that gets corrupted, but the data at the top of the stack. It works like this, but I’d like to know the reason of the corruption.

Any ideas?

UPD:
As asked in the comments, here are the actual function types:

typedef unsigned char (CALLBACK Tfr)(unsigned char);
typedef void (CALLBACK Tfw)(unsigned char,unsigned char);
typedef int (CALLBACK Tfs)(int);
typedef void (CALLBACK Tfwf)(int*,int);

All they show a similar behavior.

Unfortunately, it is not so straightforward to attach a debugger, as the code is compiled and launched by Matlab, using the LCC compiler, and there is no debugging support. Probably I will have to reproduce this problem in a standalone configuration, but it is not that easy to make it.

Answer 1

Sounds like you use MSVC, Debug + Windows + Registers. Look at the value of ESP before and after the call. If it doesn’t match then first change the calling convention in the function pointer declaration (did you do that right?) If it still doesn’t match then it is __stdcall and you haven’t guessed the arguments you need to pass correctly.

Or the function could just clobbers the stack frame, it isn’t impossible.

Posting your function pointer declaration that shows the real arguments would probably help diagnose this better.