Question

I have injected my dll into process.
How can I get Main window handle of host application?

Answer 1

The host application may have multiple ‘main windows’. To detect them, you could

1. Call GetCurrentProcessId to get the PID of the current process
2. Call EnumWindows to iterate over all toplevel windows of the desktop
3. For each window on the desktop, call GetWindowThreadProcessId to get the PID of the process which created the window
4. If the PID of the window matches the PID of your own process, memorize the window.

That gives you a list of toplevel windows created by the process which you injected your DLL into. However, please note that this approach may yield windows which have been destroyed by the time you process the constructed list of windows. Hence, when doing something with the windows, make sure to use the IsWindow function to ensure that the window at hand is still valid (this is still prone to race conditions since the window may become invalid between your call to IsWindow and actually accessing the window, but the time window is much smaller).

Here’s a C++ function implementing this algorithm. It implements a getToplevelWindows function which yields a std::vector<HWND> containing the handles of all toplevel windows of the current process.

struct EnumWindowsCallbackArgs {
    EnumWindowsCallbackArgs( DWORD p ) : pid( p ) { }
    const DWORD pid;
    std::vector<HWND> handles;
};

static BOOL CALLBACK EnumWindowsCallback( HWND hnd, LPARAM lParam )
{
    EnumWindowsCallbackArgs *args = (EnumWindowsCallbackArgs *)lParam;

    DWORD windowPID;
    (void)::GetWindowThreadProcessId( hnd, &windowPID );
    if ( windowPID == args->pid ) {
        args->handles.push_back( hnd );
    }

    return TRUE;
}

std::vector<HWND> getToplevelWindows()
{
    EnumWindowsCallbackArgs args( ::GetCurrentProcessId() );
    if ( ::EnumWindows( &EnumWindowsCallback, (LPARAM) &args ) == FALSE ) {
      // XXX Log error here
      return std::vector<HWND>();
    }
    return args.handles;
}

UPDATE: These days (about four years after I gave the answer) I would also consider traversing the list of threads of the application and then using EnumThreadWindows on each thread. I noticed that this is considerably faster in many cases.