Question

As a C# developer, I think many of us have the experience as calling unmanaged code from managed code.

What is the most difficult problem you’ve ever faced when dealing with interoperability between managed code and unmanaged code? (You had better give code examples such as C++ function signature and C# wrappers)

How do you debug the runtime error since no high level debug information provided by visual studio when dealing with the calling?

Answer 1

In general, finding an authoritative reference for P/Invoke declarations is difficult if you don’t know enough about C to get the declaration right. There are a lot of bad declarations out on the Internet. The typical traps are VB6 declarations, there are a lot of them out there, where arguments that are really int are declared as Long. Long is the 32-bit integral type in VB6, Integer is 16-bit for historical reasons.

Another wide-spread mistake is using int where IntPtr is required. These declarations work fine in 32-bit mode but misbehave in 64-bit mode. Especially troublesome are functions that take 4 or less arguments, the wrong argument type doesn’t generate the PInvokeStackImbalance MDA warning because the first 4 function arguments are passed in CPU registers. This bug tends to bite long after the author declared the job done, pretty hard to get back into it and diagnose the trouble.

The next hassle is the tedium of declaring and maintaining either P/Invoke declarations or C++/CLI wrappers. When the unmanaged API is big, you’ll end up writing a heck of a lot of them. Each of them ready to give you a hard to diagnose runtime error when you get it subtly wrong.

The truly hard interop is with COM servers that have IUnknown based interfaces. Such servers don’t have a type library so there’s no opportunity to use Tlbimp.exe to auto-generate the interop library. You have to re-declare the interfaces in your C# code and make sure you get [Guid] and the method declarations right. The lack of support for multiple inheritance requires jumping through extra hoops when the interface inherits from an intermediate IUnknown based interface. The problem that this author incorrectly calls the ‘inheritance tax’.

These kind of servers are way more common then meets the eye. Just about anything you’d want to do with the shell (explorer.exe) is IUnknown based for example. So are the Vista and Win7 extensions, ably wrapped by the Windows API Code Pack btw.

And yes, error handling. Way too much unmanaged code out there that return nothing but an unfriendly error code like E_FAIL or E_UNEXPECTED. And doesn’t implement IErrorInfo so all you got is an unhelpful generic message to look at. Or APIs that return BOOL to indicate failure, inviting a programmer to simply ignore the return value. And AccessViolation or Fatal Engine Execution Error, unmanaged code blowing up in your face without leaving any breadcrumb to find out what went wrong. This isn’t particular to C# interop, such code is hard for anybody to use.

Other than that, it usually is pretty simple to make it work. The P/Invoke marshaller and the COM interop support in .NET are truly excellent and significantly better than what’s available in other VMs.