Question

I am developing UI Automation client(c++) for metro app. I am able to get element on the metro UI.I am using Raw Tree Walker

1. getting the Automation Tree

   automation->get_RawViewWalker(&pTreeWalker);
   

2. then I am iterating and getting the element on metro app

3. Retrieving the properties of the UI element

   I have image and text item in side the list item. On click on that launch a webpage.

   list item, image and Text item doesn’t have Invoke Capability.so I am trying to get Clickable Point, so that I can click the point.I am also using Inspect.exe to see the UI element.in

   inspect.exe does show cliackable point for listitem, image and text item.but when i

   problematically trying to get it using GetClickablePoint() i am getting gotClickable = 0 and POINT field remains 0.hr = S=OK

   POINT clickable;   
   BOOL gotClickable;     
   hr = p1->GetClickablePoint(&clickable,&gotClickable);
   

I want to use the clickable point in the call to mouse_event

   VOID WINAPI mouse_event(__in  DWORD dwFlags, __in  DWORD dx, __in  DWORD dy, __in  DWORD dwData,
   __in  ULONG_PTR dwExtraInfo  );

Answer 1

Check to see if the item has a zero hwnd. This comes up often for me.

You can still click by moving the Cursor to the middle of the element and sending a mouse click event.

You can compute the location based on the bounding rectangle. You may need to convert client to screen coordinate, depending upon your situation.

The bounding rectangle part is something like:

x = left + (right - left) / 2
y = top + (bottom - top) / 2 

To convert to screen coords, you will have to use an element which does have an hwnd and apply the necessary offset.

EDIT

To place the cursor on the rectangle, I use PInvoke:

Here is the stuff that I use (C#):

    public static void Click()
    {
        User32.mouse_event(WindowsConstants.MouseEventLeftDown, 0, 0, 0, IntPtr.Zero);
        User32.mouse_event(WindowsConstants.MouseEventLeftUp, 0, 0, 0, IntPtr.Zero);
    }

    public static void RightClick()
    {
        User32.mouse_event(WindowsConstants.MouseEventRightDown, 0, 0, 0, IntPtr.Zero);
        User32.mouse_event(WindowsConstants.MouseEventRightUp, 0, 0, 0, IntPtr.Zero);

    }

    public static void DoubleClick()
    {
        User32.mouse_event(WindowsConstants.MouseEventLeftDown, 0, 0, 0, IntPtr.Zero);
        User32.mouse_event(WindowsConstants.MouseEventLeftUp, 0, 0, 0, IntPtr.Zero);
        Thread.Sleep(150);
        User32.mouse_event(WindowsConstants.MouseEventLeftDown, 0, 0, 0, IntPtr.Zero);
        User32.mouse_event(WindowsConstants.MouseEventLeftUp, 0, 0, 0, IntPtr.Zero);
    }

    public const UInt32 MouseEventLeftDown = 0x0002;
    public const UInt32 MouseEventLeftUp = 0x0004;
    public const UInt32 MouseEventRightDown = 0x0008;
    public const UInt32 MouseEventRightUp = 0x00010;


    [DllImport("user32.dll")]
    public static extern void mouse_event(UInt32 dwFlags, UInt32 dx, UInt32 dy, UInt32 dwData, IntPtr dwExtraInfo);