Question

I have a third party encryption library, which may create a MessageBox if key creation fails. The failure can be caused by bad random number generation or other rarities, and in most cases, trying again will result in success. My code will attempt key creation up to three times before deciding it failed.

Now, the issue is that the program may be used with automation. If a MessageBox is created during automation, it will block the process forever, because there’s nobody to click the ‘OK’ button.

Does anyone know of a way to catch when this message box is created and automatically close it?

Anything is fair game, as long as it’s not something that will make security suites angry. This means no hooking or code tunneling.

In summary, I need to catch when a MessageBox is created and close it. The MessageBox’s creation is outside of my control. Modifying the code at runtime is not acceptable.

Also, I’ve noticed there are some other similar questions, but they don’t have the same requirements.

EDIT: Additional note, I can find the message box via searching through all windows until I find one with a matching title and then send it a WM_CLOSE message, but I don’t think this is a great solution. I also have no guarantee that the message box has been/will be displayed, or how long after my call it will be displayed. It could display instantly, it could display 1200 ms later, or it could not display at all.

Answer 1

Just before you begin the encryption process, install a WH_CBT hook, and in its callback watch for an nCode of HCBT_CREATEWND. If you get a matching class name (‘#32770 (Dialog)’ ?) and a matching title either return a nonzero value from the callback, or if that doesn’t work post a WM_CLOSE (or a BM_CLICK to a relevant button if selecting an option is necessary). Uninstall the hook after the process for not messing with every possible dialog your application pops up.