Question

I’m trying to PInvoke into this C++ library function:

int Initialize(Callback* callback);

struct Callback
{
    //.......
    void virtual StateChanged(int state) = 0;
};

I have tried this naive C# code, but it doesn’t work.

[DllImport("CPlusPlusLibrary.dll", SetLastError = true, EntryPoint = "?Initialize@CPlusPlusLibrary@@YAHPAUCallback@1@@Z")]
public static extern int Initialize(Callback callback);

[StructLayout(LayoutKind.Sequential)]
public class Callback
{
    //....
    public delegate void IntDelegate(int state);
    public IntDelegate StateChanged(int state);
}
var callback = new Callback();
var result = Initialize(callback);

Answer 1

It is impossible to do it that way as far as I know. Methods are not “in there” as fields would be, beside this, creating a struct with virtual method will create a vtable pointer in your objects, that you are not taking into account in c# mirrored class.

What you can do is to PInvoke to method that takes a functionPointer (in C++) and pass a delegate (C#) there. You can then use this function pointer to call it from native code and your delegate will launch.

You could then change your stateChange method definition to take a Callback* as a first parameter, so when you call it from native code you can pass an object pointer which is responsible of that change and marshal it back to Callback in c#.

//Edit without having source of native dll, building a bridge between c# and c++ is what comes to my mind. This can be done with c++/cli or c++, sticking to native my idea would be something like this:

//c++ <--> new c++ dll <--> c#
struct CallbackBridge : public Callback
{
    void (*_stateChanged)(int);

    virtual void stateChanged(int state)
    {
        if (_stateChanged)
            _stateChanged(this, state);
    }
};

void* CreateCallback() { return new CallbackBridge(); }
void DeleteCallback(void* callback); { delete callback; }
void setStateChanged(void* callback, void (*ptr)(void*, int))
{
    CallbackBridge* bridge = (CallbackBridge*)callback;
    bridge->stateChanged = ptr;
}
///... other helper methods

The idea here is to treat your object as a black box (hence void* everywhere – it can be any pointer, but from c# you will just see this a a SafeHandle / IntPtr and write helper methods that you can PInvoke to to create / delete and modify objects. You can mock those virtual calls by giving your object a delegate through such method.

From c# usage could look like this: (IntPtr for simplicity, SafeHandle could be used):

IntPtr callback = CreateCallback();
SetStateChanged(callback, myCallback);
//somewhere later:
DeleteCallback(callback);

void MyCallback(IntrPtr callback, int state)
{
    int someData = SomeOtherHelperMethod(callback);
    ConsoleWrite("SomeData of callback is {0} and it has changed it's state to {1}", someData, state);
}

I know, it’s a bit clumsy for bigger objects, but without c++/cli wrapper I have never found any better way to be able to handle all those tricky cases like virtual calls etc.