Question

I am trying to interface with a Dll that implements several functions, one of which takes a null terminated string and an int, and returns a null terminated string. I have attempted to interface with the method like this:

[DllImport(dll_loc)]
[return : MarshalAs(UnmanagedType.LPStr)]
public static extern StringBuilder GetErrorMessage([MarshalAs(UnmanagedType.LPStr)]
                                                       StringBuilder message, 
                                                       int error_code);

I then attempt to call the method like this:

StringBuilder message = new StringBuilder(1000);
StringBuilder out2 = new StringBuilder(1000);
out2 = GetErrorMessage(message, res0);

However, when I attempt this, an AccessViolationException is thrown telling me I am attempting to access protected memory.

I was successful in declaring a different method as such:

[DllImport(dll_loc)]
public static extern int GetVersion([MarshalAs(UnmanagedType.LPStr)]
                                        StringBuilder version);

and calling it in the same manner, but this method won’t work for the current function call.

I also tried returning an IntPtr, since the documentation technically says that the method returns a pointer to the first character of the string buffer, but to no avail.

Does anyone have any insight into what could be going wrong here? What could be different between these two methods that is causing the dll to attempt to access memory it should not. Or, how would you recommend going about debugging this problem?

Answer 1

C functions that return a string are a memory management problem. A C string requires an array and the memory for that array needs to be released after the string is consumed. That makes such functions very hard to use in a C program, next to impossible to use with pinvoke. It is also a classic C bug, returning a pointer to a string on the stack.

The pinvoke marshaller is going to try to release the returned string, as required to avoid a memory leak. It will use CoTaskMemFree(). That does not often come to a good end, it is rare that the C code actually used CoTaskMemAlloc to allocate the memory for the array. On XP, that will cause a silent memory leak. Vista and Win7 have much stricter heap managers, they’ll invoke a debug break if an unmanaged debugger is attached. And bomb the program with an AccessViolation next.

You can avoid the automatic marshaling behavior by declaring the return type as IntPtr and marshal the string yourself. Typically with Marshal.PtrToStringAnsi(). But then you’re still faced with the task of releasing the memory. You can’t, you don’t have the handle of the CRT heap and you can’t call free().

C functions that return strings should be declared with an argument that passes a string buffer. And an argument that says how large the buffer is. Like this:

 int GetErrorMessage(int errorCode, char* buffer, size_t bufferSize);

Now it is simple, the caller can allocate the memory for the buffer and free it. And the C function simply copies the string into the buffer. The return value can be used to tell how many characters were copied, or indicated that a larger buffer is needed. Don’t skimp on the bufferSize argument, a buffer overflow is deadly and invokes the dreaded FatalExecutionEngineError exception, an exception that is as undebuggable as AV since the GC heap corruption doesn’t get detected until much later. You use a StringBuilder on the C# side, suitably initialized with a non-zero capacity. The value of bufferSize.