Question

My application uses some APIs like GetProcAddress and CreateProcess that cause sometimes antiviruses to flag it as malicious even though it is not.

What I am trying to do is check whether a specific API is being monitored or hooked and if it is then I won’t call that part of the code.

How do I check whether a certain API is hooked?

This is a Windows application written in C.

Thanks.

Answer 1

On win32 there are no offical methods to detect and/or place hooks (besides the SetWindowsHookEx() (http://msdn.microsoft.com/en-us/library/windows/desktop/ms644990) et al functions which only cover a very small set of functionality).

Detecting a hook depends on how the hook was applied.

There are two popular methods to place a hook:

1. Import/Export table patching
2. Code overwriting

For details (pros/cons) on the different methods to place hooks please consider reading here http://help.madshi.net/ApiHookingMethods.htm.

Each method of hooking requieres a different approach to detect it.

For methods to detect hooks placed as mentioned above please look under “ApiHookCheck Algorithm” here http://www.security.org.sg/code/apihookcheck.html. There are sample sources available on this page, which I did not test.