Question

I’m having a little trouble trying to get WarpPointer to work out appropriately for me. The scenario I’m trying to do is akin to recording a user’s clicks and then replaying them on the screen. I’ve been able to record the screen motions and clicks without any issues; however, I’m stumbling when it comes to replaying what the user recorded.

def MoveCursorAndClick(self, e):
    # get the current screen position
    sp = self.GetPosition()
    p = self.getNextPosition() # returned in absolute x,y from screen rec
    self.WarpPointer(p.x - sp[0], p.y - sp[1])
    # wx.PostEvent(EVT_LEFT_DOWN, self.GetId())

Obviously, the above code is incorrect because WarpPointer appears to work within the window only and not the full screen. Attempting to base the coordinated off of the functioning window as well proves to be troublesome because they always seem to be off in the Y direction by about 50 pix. My only current assumption is I’m making this harder than it needs to be and there’s a better why I’m not seeing due to my inexperience with wxpython.

Additionally, the busted PostEvent line is the only thing I’ve managed to google across that looks like the right thing for sending a mouse click to something that may or may not be another window. Is that even the right direction?

Updated
As mentioned below, there’s not a good cross platform way to accomplish this. Here’s what I did (found various places on the internet):

def ClickScreenPositon(self, x, y):
    win32api.SetCursorPos((x,y))
    win32api.mouse_event(win32con.MOUSEEVENTF_LEFTDOWN,x,y,0,0)
    win32api.mouse_event(win32con.MOUSEEVENTF_LEFTUP,x,y,0,0)

Answer 1

I don’t think you can do this in wxPython alone. wx.Window.WarpPointer() can only move the mouse to an area controlled by that window, as you’ve discovered. Similarly, you’re wx.EVT_LEFT_DOWN call won’t work because you’re setting the ID to the window, meaning it’s the window that was clicked, not a click at (x,y).

After some light Googling and I found this article on Unit Testing in wxPython that breifly talks about testing at the GUI level. This isn’t exactly what you’re trying to do, but it is conceptually similiar. They suggest using an external library to manipulate the interface. The main problem they note about this approach is that most of the tools for simulation GUI interaction are under corporate licenses and/or platform dependent.

That article suggests using pywinauto for Windows systems. For your purposes, pywinauto.controls.HwndWrapper seems to have the methods you require; MoveMouse() and Click(). If you aren’t using Windows or if you need cross-platform support you will need to research different libraries.

Basically, you will need to install and import pywinauto into your wxPython application and call its methods instead of wxPython’s. As long as cross-platform compatibility is not an issue I think this is your simplest, best solution.