Question

I have a file pointer exported from a dll, which is initialized(fopen) by the application and then used(fprintf) inside the dll.

The problem is fprintf will throw an exception.

DLLFile.c

#define BUILD_FOO
#include "API.H"

File *pFile;

void exportedFunction()
{
   fprintf(pFile,"This will result in an exception\n");//<-This print will crash
}

API.H

#ifdef BUILD_FOO
#    define FOOAPI __declspec(dllexport)
#else
#    define FOOAPI __declspec(dllimport)
#endif

FOOAPI  extern File *pFile;
FOOAPI  void exportedFunction();

APLICATION.C

#undef BUILD_FOO
#include "API.H"
void main()
{
pFile = fopen("path_to_folder","wt");
fprintf(pFile , "This print will work"); // <- This will be printed ok
exportedFunction(); 
}

1 From the debugging I’ve done, this is what I saw:

Inside the application, fopen() assigns for pFile an element from _iob[].

In the DLL when fprintf is called, it is checked that pFile is part of the _iob[], but the _iob[] from the application seems not to be the same with the one in the DLL(they have different addresses).

2 I have the same use case(with the same application) and another somewhat similar DLL, and everything works ok there(the _iob[] is at the same place in the application and DLL).

Answer 1

This is likely being caused by your application and your DLL disagreeing on which version of the C runtime they’re using. Unless they’re both compiled against the exact same version of the C runtime, all bets are off, and you can’t call CRT functions from one using the data from another or vice-versa.

The safest way to avoid this problem is not to pass FILE* pointers across DLL boundaries. That way, any interaction with a FILE* will always happen using the same version of the CRT, and there’s no danger of any mismatches. So your DLL should not expose a FILE* variable; instead it should be some opaque type, and all operations on the variable need to happen in the same module.

For example:

// API.h
FOOAPI void set_file(void *file);
FOOAPI void set_fprintf_callback(int (*my_fprintf)(void *, const char *, ...));
FOOAPI void exportedFunction();

// DLLFile.c
void *pFile;  // Not exported
int (*fprintf_callback)(void *, const char *, ...);  // Not exported

FOOAPI set_file(void *file)
{
    pFile = file;
}

FOOAPI set_fprintf_callback(int (*my_fprintf)(void *, const char *, ...))
{
    fprintf_callback = my_fprintf;
}

FOOAPI exportedFunction()
{
    // Call back into the application to do the actual fprintf
    fprintf_callback(pFile, "This should not crash");
}

// Application.c
int mydll_fprintf(void *pFile, const char *fmt, ...)
{
    va_list ap;
    va_start(ap, fmt);
    int result = vfprintf((FILE *)pFile, fmt, ap);
    va_end(ap);

    return result;
}

int main()
{
    FILE *pFile = fopen(...);
    set_file(pFile);
    set_fprintf_callback(&mydll_fprintf);
    exportedFunction();

    return 0;
}