Question

I have an one question when I use C# DllImport C++ dll, I use the visual studio 2010 & checked the “Enable unmanaged code debugging”, when it’s running, always show the message “Buffer overrun detected! … A buffer overrun has been detected which has corrupted the program’s internal state. The program cannot safely continue execution and must now be terminated.”

This dll is Third-party vendors to provide, and they says it’s no error to run this dll, how can i fixed it?

My M++ dll function is,

int  avc_to_avi_convert(char* chPath_avc, char* chPath_avi, int pPrivate, PROGRESS_CALLBACK progress_callback);

void avc_to_avi_close(int* pFd_avi);

typedef void (*PROGRESS_CALLBACK)(int iPercent, int pPrivate);

And i am use it in my C# dllimport like this :

public delegate void PROGRESS_CALLBACK(int _iPercent, int _pPrivate);

[DllImportAttribute(@"..\xxx.dll", EntryPoint = "avc_to_avi_convert", CallingConvention = CallingConvention.Cdecl, CharSet = CharSet.Unicode)]

public static extern unsafe int avc_to_avi_convert([MarshalAs(UnmanagedType.LPStr)] StringBuilder _inputAVC, [MarshalAs(UnmanagedType.LPStr)] StringBuilder _ouputAVI, int pPrivate, PROGRESS_CALLBACK _pg);

[DllImportAttribute(@"..\xxx.dll", EntryPoint = "avc_to_avi_close", CallingConvention = CallingConvention.Cdecl, CharSet = CharSet.Unicode)]

public static extern unsafe void avc_to_avi_close(int pFd_avi);

private void button1_Click(object sender, EventArgs e)
{
    int i = 0;
    StringBuilder inputFile = new StringBuilder(Application.StartupPath + @"\avc\abc.avc");//(@"C:\avc\abc.avc");
    StringBuilder outputFile = new StringBuilder(Application.StartupPath + @"\avi\abc.avi");//(@"C:\avi\abc.avi");

    if (avc_to_avi_convert(
        inputFile,
        outputFile,
        1,
        progress_func) > 0) { 

    }
}

public void progress_func(int iProgress, int pPrivate)
{
     if (iProgress == 100)
     {
         //success
     }
     else if (iProgress == -1)
     {
         //convert error
     }
     else if (iProgress == -2)
     {
         //Not enough disk space
     }
     else
     {
         //update progress
     }
}

I changed my code to,

[DllImportAttribute(@"..\xxx.dll", EntryPoint = "avc_to_avi_convert", CallingConvention = CallingConvention.StdCall, CharSet = CharSet.Unicode)]

public static extern unsafe int avc_to_avi_convert([MarshalAs(UnmanagedType.BStr)] String _inputAVC, [MarshalAs(UnmanagedType.BStr)] String _ouputAVI, int pPrivate, PROGRESS_CALLBACK _pg);

[DllImportAttribute(@"..\xxx.dll", EntryPoint = "avc_to_avi_close", CallingConvention = CallingConvention.StdCall, CharSet = CharSet.Unicode)]

public static extern void avc_to_avi_close(ref int avi);

And ran it, and I still get the same error.

Answer 1

1.) Are you sure about the calling convention? Try CallingConvention.StdCall. Read this:

http://blogs.msdn.com/b/adam_nathan/archive/2003/05/21/56690.aspx

2.) Try a different values for the CharSet attribute and use String instead of StringBuilder for the path arguments. This is a good reference:

http://msdn.microsoft.com/en-us/library/s9ts558h.aspx

3.) Also, the avc_to_avi_close method should look like this:

[DllImportAttribute("xxx.dll", EntryPoint="avc_to_avi_close")]
public static extern void avc_to_avi_close(ref int avi);

4.) Other things you my want to try:

• Use unusual combinations of CharSet and MarshalAs. For example you know from the link above that Unicode should be used with LPWStr and Ansi with LPStr but you can of course try other combinations.
• Your extern methods need not to be marked unsafe.

If you have exhausted all other options, why don’t you try to write a C++ program that is analogous to your C# code:

• If it crashes you proved that the error is in the library.
• If it doesn’t crash you know the error is in your use of PInovke, and you can go through all the 12+ combinations from my answer one by one.