Question

I use PInvoke to access a C++ library from c# code. But I have problems using the following function:

UINT32 myfunc(IN char * var1, IN UINT32 var2, INOUT UINT16 * var3, OUT UINT8 * var4)

documentation says:

return parameter is error code

var1 is a string

var2 is an integer

var3 is a pointer to an integer (the value will be the size of var4, the value will be changed by the function)

var4 is a pointer to a buffer that will be filled by the function

When I use

[DllImport("mydll.dll", CallingConvention = CallingConvention.Cdecl)]
public static extern int myfunc(string var1, int var2, out int var3, 
    out byte[] var4);

The function gives the following exception:

Managed Debugging Assistant ‘FatalExecutionEngineError’ has detected a problem in ‘C:\Users…\Documents\Projects…\bin\Debug…vshost.exe’.

Additional Information: The runtime has encountered a fatal error. The address of the error was at 0x74292e44, on thread 0x2edc. The error code is 0xc0000005. This error may be a bug in the CLR or in the unsafe or non-verifiable portions of user code. Common sources of this bug include user marshaling errors for COM-interop or PInvoke, which may corrupt the stack.

If there is a handler for this exception, the program may be safely continued.

I tried to work with parameters to see if I can get anything and I changed charset as below:

[DllImport("mydll.dll, CallingConvention = CallingConvention.Cdecl", CharSet = CharSet.Unicode)]
public static extern int myfunc(string var1, int var2, out int var3, out byte[] var4);

In this case, it does not give FatalExecutionEngineError, instead the function returns without error. But the return value indicates that an internal error hass occurred in the function. (I don’t know why the charset change prevents the FatalExecutionEngineError.)

I am not sure what I am doing wrong, please help me.

Answer 1

The p/invoke declaration should look like this:

[DllImport("mydll.dll", CallingConvention = CallingConvention.Cdecl)]
public static extern int myfunc(
    string var1, 
    uint var2, 
    ref ushort var3, 
    byte[] var4
);

And then you call it like this:

string var1 = "...";
uint var2 = ...;
ushort var3 = 1024;//use whatever length buffer you want to pass
byte[] var4 = new byte[var3];
int retval = myfunc(var1, var2, ref var3, var4);
Array.Resize(ref var4, var3);

The way that the native code works is that it expects the calling code to allocate the buffer, var4. The calling code says how long the buffer is in var3.

Then when the native code returns, var3 contains the actual length of the buffer. Or perhaps the number of bytes that were copied. The former is more useful. Assuming the former then you could perhaps write it like this:

string var1 = "...";
uint var2 = ...;
ushort var3 = 0;
int retval = myfunc(var1, var2, ref var3, null);
byte[] var4 = new byte[var3];
int retval = myfunc(var1, var2, ref var3, var4);

You should be checking return values etc.

I can’t be 100% sure exactly how to call the function because there’s no enough information in your question. At least the p/invoke signature now matches your C++ code.