Question

I am debugging some other programmer’s source code of a Windows Media Player plugin. This plugin causes WMP to crash sometimes, and sometimes it takes really long time to open plugin settings window. The problem occurs only when opening settings window while music is being played back. It opens without issues if the player is stopped.

While looking through the code and debugging, I have came to the line of code which seems to be the cause of the problems.

The property page has the following member variable:

CComPtr<IDsp_plugin> m_pDsp_plugin;

and the property page at initialization calls get_text method of the COM object:

unsigned char * txt = NULL;
//m_pDsp_plugin is a valid pointer to IDsp_plugin
HRESULT res = m_pDsp_plugin->get_text(&txt);

At this moment hres is “0x80010105: The server threw an exception.” and Visual Studio Debug output shows “First-chance exception at 0x764efbae in wmplayer.exe: 0x80010105:

get_text method is defined as follows:

in Dsp_plugin.idl

interface IDsp_plugin : IUnknown
{
    HRESULT get_text([out] unsigned char* *pVal);
...

in Dsp_plugin.h

class ATL_NO_VTABLE CDsp_plugin : 
    public CComObjectRootEx<CComMultiThreadModel>,
    public CComCoClass<CDsp_plugin, &CLSID_Dsp_plugin>,
    public IDsp_plugin,
    public IMediaObject,
    public IWMPPluginEnable,
    public ISpecifyPropertyPages
{

    STDMETHOD(get_text)(unsigned char* *txt);
...

and finally the method itself which throws this exception:
Dsp_plugin.cpp

STDMETHODIMP CDsp_plugin::get_text (unsigned char* *txt)
{
   ... // some code for copying a valid string from somewhere to char* y
   // 11 bytes of memory for y was allocated using malloc(10+1);
   // y contains a valid C string here, tested with debugger and passing to OutputDebugStringA

   *txt = (unsigned char*)(y); // This line executes normally, but at the end the caller gets "The server threw an exception." and WMP starts behaving weirdly. 
                               // If I comment it out, the caller gets S_OK and there are no any issues with WMP.

   return S_OK;
}

The COM DLL is compiled with setting “Use Unicode Character Set”.

I am not experienced COM programmer, but passing strings as unsigned char** seems unusual to me, I have seen mostly BSTR or VARIANT when dealing with COM.

Maybe some COM guru can explain, why this exception happens and can it be possibly fixed just by converting methods to using BSTR* and SysAllocString/SysfreeString instead of unsigned char**/malloc/free ?

Answer 1

Put simply, COM doesn’t know how to pass around pointers of type unsigned char *. The default marshalling rules are applied (since the interface definition doesn’t specify any parameter attributes), and, if I’m interpreting this correctly, COM marshals the outer pointer itself txt properly, but treats *txt as a pointer to a single unsigned char, not a string.

This may still work if the caller and callee happen to be in the same apartment; from the sounds of it, they’re not.

The easiest solution is simply to make the parameter a BSTR *. COM has special handling for BSTR which will ensure it’s passed correctly.