Question

I have a dll function that takes BSTR parameters. These are cast as char* before being used for other things.

When the dll is called from VBA code this works. However, when it is called from C# code, only the first character is pointed to.

Both of these are Excel addIns for pre-2007 and 2007+ versions of Office, which call into a faster C++ AddIn. They call it directly, not through Excel.

The VBA function declaration:

Private Declare Function Test Lib "ExcelAddIn.xll" (ByVal param As String) As String

The C# function declaration:

[DllImport("ExcelAddIn.xll", CharSet=CharSet.Ansi)]
[return:MarshalAs(UnmanagedType.BStr)]
private static extern string Test([MarshalAs(UnmanagedType.BStr)] string param);

When debugging the dll and watching the input BSTR values, they appear to be correct from both; just the C# one only casts the first character.

Charset=CharSet.Unicode makes no difference.

C++ code.

BSTR __stdcall Test(BSTR param)
{
char* Param= "";

if(param!= NULL)
    Param= (char*)param;

    return OtherClass.DoOtherStuff(Param);
}

Answer 1

The reason why has to do with the way you are marshalling the data.

The VB declaration contains no annotations on the string parameter and hence it will marshal as a normal char* parameter. The CLR can do no type verification for pinvoke and essentially puts a char* in a slot which expects a BSTR. Your code though immediately casts it to a char* and it fixes the marshalling problem.

In the C# example though you explicitly said to marshal the string as a BSTR. A BSTR is actually a string which uses wchar under the hood. By doing a simple cast to a char* you are essentially casting a wchar* to a char* which explains why you only see the first character.

The easiest solution is to just have the Test method take a char* parameter and remove the marshalling annotation on the C# version.