Question

I have a .NET 3.5 winforms app that calls a method in a VB6 ActiveX DLL, which brings up a dialog box. Everything worked fine for almost 10 years (first as a .NET 1.1 app, then 2.0, then 3.5, all on Windows XP). Now that the customer has upgraded to Windows 7, I am experiencing a System.AccessViolationException: Attempted to read or write protected memory. This is often an indication that other memory is corrupt.

The structure of the call is like this:

.NET Code

try {
  VB6Object obj = new VB6Object();
  obj.DoStuff();
}
catch (Exception ex) {  // handle }

VB6 code in VB6Object class

Public Sub DoStuff()
    on error goto errHandle
    RetrieveData();

    Call frmPopup.Show(vbModal)

exitPoint:
    exit sub
errHandle: 
    ' handle error
End Sub

The problem starts in the VB6 code on this line: Call frmPopup.Show(vbModal). The error is propagated to the .NET code (error handlers in the VB6 code are completely ignored) where I handle the error. I thought the problem might be the Load event of the frmPopup, but the code does not ever reach the Load event (i have tracing statements all over the place, so that’s how I know).

The interesting part is that the second time I make the call, it does not fail at all, but the form is missing a 3rd party grid (ActiveX DevExpress Quantum Grid), so I am assuming that the grid is probably not compatible somehow with Windows 7.

Everything that I have described happens when I simply run the application as an EXE or from VS2008 IDE, so the VB6 ActiveX DLL is simply a reference. If I use COM indirection and run the VB6 code in the IDE, everything works perfectly fine.

I’ve tried running things as an Administrator, but nothing changed.

Where can I go next to troubleshoot this issue?

Answer 1

Turn DEP off. Older Ax Controls tend to use funky subclassing thunks (other than ATL/MFC ones) that no one cares to set the NX flag off.