Question

How to call a post-implemented function (e.g., function pointers or virtual functions implemented in the consuming application) with a complex return type from a DLL function?

I tried the following scheme but got errors.

Test.h:

#ifdef _DLL_IMPL
#define DLL_EXPORT __declspec(dllexport)  
#else  
#define DLL_EXPORT __declspec(dllimport)  
#endif
typedef string (*test)(void);
extern DLL_EXPORT test __test;
DLL_EXPORT int launch();

Test.cpp:

#define _DLL_IMPL
#include "Test.h"
test __test = 0;
int launch()
{
    string a = __test();
    return 0;
}

And the consuming application goes like:

Main.cpp:

#include "Test.h"
#pragma comment(lib, "Test.lib")
string test_impl()
{
    string a = "test";
    return a;
}
int main(int args, char** argv)
{
    __test = &test_impl;
    return launch();
}

And I got a subsequent error message:

Windows has triggered a breakpoint in TestRun.exe.

This may be due to a corruption of the heap, which indicates a bug in
TestRun.exe or any of the DLLs it has loaded.

This may also be due to the user pressing F12 while TestRun.exe has focus.

The output window may have more diagnostic information.

I don’t know what exactly was going on. Error also occurred when I tried a return type of a char pointer, which would be created in the consuming application using a new operator and would be freed in a DLL function using a delete[] operator.

Can someone explain the reason why the error happens and suggest me some solution to this scheme? Thank you!

Answer 1

Passing C++ objects between exe and dll is really not a good idea, more so if the object is based on a template class and/or has inlined methods, and even more so if the object allocates memory internally. If you need that kind of interface between your application and the library then I recommend that you switch your dll to a static library, then you’ll avoid most issues.

If you need to keep the dll as a dll, then I recommend that you only pass native types between exe and dll. In your example, switching all the usages of string to char* will likely address the crashes.

Also take good advice from Jim Rhodes and declare an explicit calling convention, even if you found that that was not the problem in this particular case.