Question

I need to call a c++ method via dllimport. Everything is working fine as long as there are not the following two parameters in the c++ method:

const void * key,
void * out

I think I need to marshal them. But how does it work?
The key should be a pointer to a byte array and the out parameter is also a byte array with a length of 16.

After trying what Jcl suggests, I have the following:

Using the first method (use out byte[] outprm), the program crashes without error (when reaching the call point).
But using the second way (from Jcl’s comment), I have the following:

[DllImport(@"MyDLL.dll", SetLastError = true)]
public static extern void MyMethod(IntPtr key, out IntPtr outprm); 

private void button1_Click(object sender, EventArgs e)
    {            
        byte[] hash = new byte[16];
        byte[] myArray = new byte[] { 1, 2, 3 };
        IntPtr outprm = Marshal.AllocHGlobal(hash.Length);
        IntPtr key = Marshal.AllocHGlobal(myArray.Length);
        Marshal.Copy(myArray, 0, key, myArray.Length);
        MyMethod(key, out outprm);
        Marshal.Copy(outprm, hash, 0, 16);
        Marshal.FreeHGlobal(key);                       
    }

Now there is no error when calling MyMethod. I just get the following error when I try to copy the data back: AccessViolationException

It said I want to write into protected memory. The dll and the c# software is x64 (and needs to be x64). Maybe this i the reason?

Answer 1

Use IntPtr for both key and out, something like:

[DllImport ("whatever.dll")]
public static extern void MyMethod(IntPtr key, IntPtr outprm);

To make key be an IntPtr, taking myArray being a byte array:

IntPtr key = Marshal.AllocHGlobal(myArray.Length);
Marshal.Copy(myArray, 0, key, myArray.Length);
// ... call your function ...
Marshal.FreeHGlobal(key );

To get the 16 byte array from outprm:

IntPtr outprm;
MyMethod(key, outprm);
byte [] myArray = new byte[16];
Marshal.Copy(outprm, myArray, 0, 16);