Question

i discovered today that Windows 7 comes with a very impressive MathPanel utility, for performing handwriting recognition of equations:

Which is fine. (Here i’ve entered the formula for the part of the sRGB color space gamma conversion)

But now i don’t seem to be able to do anything with it.

There is an Insert button. i would assume that clicking Insert would insert it into the application that is active behind it (much like the On-Screen Keyboard works):

Except i assume it would operate as a Paste operation.

i can find no information in the help on what is required by an application to make it work. There is no mention of any special API some software must support.

Nor can i find any information on MSDN about what special API is required to accept the insertion of an equation.

What API, registration, callback, listener, message, COM Object do i have to implement so that i will receive MathPanel input?

The only reason i mention MathML is because an answer on SuperUser mentioned MathML:

Theoretically, any app that supports MathML (Mathematical Markup Language) can be used with the Windows 7 Math Input Panel. The Math Input Panel only works with programs that support MathML. Here are a few such apps: StarOffice, OpenOffice, Opera and Maple.

Well how do i make my program support MathML?

As far as i know MathML is a markup language; not a Windows API. It would be like saying, “How do i make my program support HTML?” Html is text, and you can paste it anywhere.

MathPad refuses to paste unless i “support” MathML?

---

Update

Inspecting the IDataObject on the clipboard after clicking Insert, i see two formats available (neither of which are text, which explains why i do not get any markup):

Format 1:

     CLIPFORMAT cfFormat: "MathML Presentation" (49839)
PDVTargetDevice ptd:      0x00000000
          DWORD dwAspect: DVASPECT_CONTENT
          DWORD lindex:   -1
          DWORD tymed:    1  (TYMED_HGLOBAL)

Format 2:

     CLIPFORMAT cfFormat:"MathML" (49838)
PDVTargetDevice ptd:      0x00000000
          DWORD dwAspect: DVASPECT_CONTENT
          DWORD lindex:   -1
          DWORD tymed:    1  (TYMED_HGLOBAL)

So at least now i have some clipboard formats:

• “MathML Presentation”
• “MathML”

i still cannot find anything on MSDN about either clipboard format.

Answer 1

Spying on messages sent to my window, looks like the Math Input Panel application sends a Ctrl+V:

• WM_KEYDOWN (0x11) VK_CONTROL
• WM_KEYDOWN (0x56) V key
• WM_CHAR (0x16)
• WM_KEYUP (0x11) VK_CONTROL
• WM_KEYUP (0x56) V key

So you need to recognize that someone’s trying to hit Ctrl+V. Then you must extract the contents.

First register the three clipboard formats:

Handle CF_MathML_Presentation = RegisterClipboardFormat("MathML Presentation");
Handle CF_MathML_Content = RegisterClipboardFormat("MathML Content");
Handle CF_MathML = RegisterClipboardFormat("MathML");

Note: Appendix B of the W3C’s Mathematical Markup Language (MathML) Version 3.0 documents the Windows clipboard format names to be registered:

• Generic MathML Windows Clipboard Name: MathML
• Presentation MathML Windows Clipboard Name: MathML Presentation
• Content MathML Windows Clipboard Name: MathML Content

Then get a handle on the IDataObject in the clipboard:

IDataObject dataObject;
OleGetClipboard(dataObject);

Then enumerate all the formats, looking for the one you like:

IEnumFORMATETC enum;
dataObject.EnumFormatEtc(DATADIR_GET, out enum);

String mathXml = "";

foreach (FormatEtc format in enum)
{
    if (format.cfFormat = CF_MathML_Presentation) ||
       (format.cfFormat = CF_MathML_Content) ||
       (format.cfFormat = CF_MathML)
    {
        //We know how to handle these formats:
        STGMEDIUM medium;
        dataObject.GetData(format.cfFormat, out medium);

        mathXml = GetStringFromStorageMedium(medium); //handles all the nasty HGlobal/IStream/IStorage nonsense
    }
}

ShowMessage(mathXml); //tada!

---

Microsoft also allows you to program the Math Input COM object:

//Create the COM object
IMathInputControl mathInputControl = CreateComObject(CLSID_MathInputControl);
mathInputControl.Show();

You can then create an object that receives the notification events:

class MathEvents : _IMathInputControlEvents
{
    public HRESULT Insert(String mathXml)
    {
       //Notifies the event handler when the Insert button is clicked.
       MessageBox.Show(mathXml);
       return S_OK;
    }

    public HRESULT Clear()
    {
       //Notifies the event handler when the Clear button is clicked.      
       return S_OK;
    }

    public HRESULT Close()
    {
       //Notifies the event handler when the Close button is clicked.
       return S_OK;
    }

    public HRESULT PaintHRESULT Paint(LONG_PTR hdc, LONG Left, LONG Top, LONG Right, LONG Bottom, LONG Element, LONG State)
    {
       //Notifies the event handler when the buttons and background of the control require painting.
       return S_OK;           
    }

The missing ingredient is how to give mathInputControl a reference to our callback object.

That’s super-secret complicated COM code, involving ConnectionPointContainer, andAdvise`, which cannot be done from C#.

But you don’t need to, you can just use Ctrl+V.