Question

So I’ve been trying to use CGPostMouseEvent, and CGEventPostToPSN to send a mouse click to a mac game, and unfortunately have been very unsuccessful.

I was hoping someone may be able to help me think of this differently, or realize what I’m missing. Google hasn’t been much help.

My guess is that it’s because I’m trying to send a click event to a game window (openGL), vs. a normal window.

Here is another example of what I’m trying to send:
CGEventRef CGEvent;
NSEvent *customEvent;
NSPoint location;
location.x = 746;
location.y = 509;

    customEvent = [NSEvent mouseEventWithType: NSLeftMouseDown
                                     location: location
                                modifierFlags: NSLeftMouseDownMask
                                    timestamp: time(NULL)
                                 windowNumber: windowID
                                      context: NULL
                                  eventNumber: 0
                                   clickCount: 1
                                     pressure: 0];

    CGEvent = [customEvent CGEvent];
    CGEventPostToPSN(&psn, CGEvent);

Interestingly enough, I can move the mouse fine (CGDisplayMoveCursorToPoint(kCGDirectMainDisplay, clickPt);), I just can’t send any clicks :/

Any help would be greatly appreciated.

Edit: Here is what is strange, once I move the mouse using CGDisplayMoveCursorToPoint, I actually have to physically move my mouse up or down a hair before I can even click, which is odd. The game doesn’t accept any input unless I move it up/down (and the pointer then changes).

Thanks!

Answer 1

Well what you are try to build is “bot” or “robot” which basically sends commands in an orderly fashion to a game. Basically it will play for you as you are afk. This is great for games that force you to play to harvest minerals, commodities or whatever gives you money to advance in the game. Which is really kind of boring. I have successfully done this for a popular game, although i cannot mention the game as it breaks the user agreements which all these type of games have against “bots”. So beware of what you are doing, as it may break your user agreement for many MMPG. But i post this successfully here because, the Mac has less bots available, none that i have been able to research, vs the PC which i have found many. So to level the playing field.. here is the code. I recommend to compile it as command line, and execute the macro in AppleScript (were the logic will reside on how to mimic the games click mouses, movements and send keys, basically your AI.

1.- First you need to run class that will get your psn “process serial number” which all games have. Basically what Thread it is running at. You can find out the name of the process in the utility in the Mac called “Activity Monitor”. This can also be done easily in AppleScript.
Once you have the name, this class will locate and give you back its psn.

#import <Cocoa/Cocoa.h>
#include <Carbon/Carbon.h>
#include <stdio.h>

@interface gamePSN : NSObject 
{
  ProcessSerialNumber gamePSN;
  ProcessInfoRec gameProcessInfo;
  pid_t gameUnixPID;    
}

- (ProcessSerialNumber) gamePSN;
- (ProcessInfoRec) gameProcessInfo;
- (pid_t) gameUnixPID;  
- (void) getPSN;
@end

@implementation gameSN

- (ProcessSerialNumber) gamePSN  { return gamePSN; }
- (ProcessInfoRec) gameProcessInfo { return gameProcessInfo; }
- (pid_t) gameUnixPID; { return gameUnixPID; }

- (void) getPSN
{
    auto OSErr osErr = noErr;
    auto OSErr otherErr = noErr;
    auto ProcessSerialNumber process;
    auto ProcessInfoRec procInfo;
    auto Str255 procName;     
    auto FSSpec appFSSpec;
    auto char cstrProcName[34];
    auto char one ='G';   // FIRST CHARCTER OF GAME PROCESS NAME THESE NEED TO BE CHANGED AS I PUT IN FAKES
    auto char two ='A';   // SECOND CHARACTER OF GAME PROCESS NAME THESE NEED TO BE CHANGED AS I PUT IN FAKES
    auto char three = 'M';  // THIRD  CHARACTER OF GAME PROCESS NAME THESE NEED TO BE CHANGED AS I PUT IN FAKES
    auto unsigned int size;

    process.highLongOfPSN = kNoProcess;
    process.lowLongOfPSN  = kNoProcess;
    procInfo.processInfoLength = sizeof(ProcessInfoRec);
    procInfo.processName = procName;
    procInfo.processAppSpec = &appFSSpec;

    while (procNotFound != (osErr = GetNextProcess(&process))) {

    if (noErr == (osErr = GetProcessInformation(&process, &procInfo))) {
        size = (unsigned int) procName[0];
        memcpy(cstrProcName, procName + 1, size);
        cstrProcName[size] = '\0';

        // NEEDS TO MATCH THE SIGNATURE OF THE GAME..FIRST THREE LETTERS
        // IF YOU CANT FIND IT WITH THE ACTIVITY MONITOR UTILITY OF APPLE MAC OS
        // THEN RUN THIS SAME CLASS WITH AN NSLOG AND IT WILL LIST ALL YOUR RUNNING PROCESSES.

        if (    (((char *) &procInfo.processSignature)[0]==one) && 
                (((char *) &procInfo.processSignature)[1]==two) && 
                (((char *) &procInfo.processSignature)[2]==three) && 
                (((char *) &procInfo.processSignature)[3]==two))
        {
            gamePSN = process;
            otherErr = GetProcessInformation(&gamePSN, &gameProcessInfo);
            otherErr = GetProcessPID(&process, &gameUnixPID);
        }

        }

    }
}

Once you have this process number it is easy to send key events as well as mouse events. Here is the mouse event clicks to send.

//  mouseClicks.h
//  ClickDep
//  Created by AnonymousPlayer on 9/9/11.


#import <Foundation/Foundation.h>

@interface mouseClicks : NSObject

- (void) PostMouseEvent:(CGMouseButton) button eventType:(CGEventType) type fromPoint:(const CGPoint) point;
- (void) LeftClick:(const CGPoint) point; 
- (void) RightClick:(const CGPoint) point;
- (void) doubleLeftClick:(const CGPoint) point;
- (void) doubleRightClick:(const CGPoint) point;

@end

/
//  mouseClicks.m
//  ClickDep
//  Created by AnonymousPlayer on 9/9/11.v


#import "mouseClicks.h"

@implementation mouseClicks

- (id)init
{
    self = [super init];
    if (self) {
        // Initialization code here if you need any.
    }

    return self;
}

- (void) PostMouseEvent:(CGMouseButton) button eventType:(CGEventType) type fromPoint:(const CGPoint) point;
{
    CGEventRef theEvent = CGEventCreateMouseEvent(NULL, type, point, button);
    CGEventSetType(theEvent, type);
    CGEventPost(kCGHIDEventTap, theEvent);
    CFRelease(theEvent);
}

- (void) LeftClick:(const CGPoint) point;
{
    [self PostMouseEvent:kCGMouseButtonLeft eventType:kCGEventMouseMoved fromPoint:point];
    NSLog(@"Click!");
    [self PostMouseEvent:kCGMouseButtonLeft eventType:kCGEventLeftMouseDown fromPoint:point];
    sleep(2);
    [self PostMouseEvent:kCGMouseButtonLeft eventType:kCGEventLeftMouseUp fromPoint:point];
}

- (void) RightClick:(const CGPoint) point; 
{
    [self PostMouseEvent:kCGMouseButtonRight eventType:kCGEventMouseMoved fromPoint:point];
    NSLog(@"Click Right");
    [self PostMouseEvent:kCGMouseButtonRight eventType: kCGEventRightMouseDown fromPoint:point];
    sleep(2);
    [self PostMouseEvent:kCGMouseButtonRight eventType: kCGEventRightMouseUp fromPoint:point];
}

- (void) doubleLeftClick:(const CGPoint) point;
{ 
    [self PostMouseEvent:kCGMouseButtonRight eventType:kCGEventMouseMoved fromPoint:point];
    CGEventRef theEvent = CGEventCreateMouseEvent(NULL, kCGEventLeftMouseDown, point, kCGMouseButtonLeft);  
    CGEventPost(kCGHIDEventTap, theEvent); 
    sleep(2);
    CGEventSetType(theEvent, kCGEventLeftMouseUp); 
    CGEventPost(kCGHIDEventTap, theEvent); 

    CGEventSetIntegerValueField(theEvent, kCGMouseEventClickState, 2);  

    CGEventSetType(theEvent, kCGEventLeftMouseDown);  
    CGEventPost(kCGHIDEventTap, theEvent);  
    sleep(2);
    CGEventSetType(theEvent, kCGEventLeftMouseUp); 
    CGEventPost(kCGHIDEventTap, theEvent); 
    CFRelease(theEvent); 
}

- (void) doubleRightClick:(const CGPoint) point;
{ 
    [self PostMouseEvent:kCGMouseButtonRight eventType:kCGEventMouseMoved fromPoint:point];
    CGEventRef theEvent = CGEventCreateMouseEvent(NULL, kCGEventLeftMouseDown, point, kCGMouseButtonRight);  
    CGEventPost(kCGHIDEventTap, theEvent); 
        sleep(2);
    CGEventSetType(theEvent, kCGEventRightMouseUp); 
    CGEventPost(kCGHIDEventTap, theEvent); 

    CGEventSetIntegerValueField(theEvent, kCGMouseEventClickState, 2);  

    CGEventSetType(theEvent, kCGEventRightMouseDown);  
    CGEventPost(kCGHIDEventTap, theEvent);
        sleep(2);
    CGEventSetType(theEvent, kCGEventRightMouseUp); 
    CGEventPost(kCGHIDEventTap, theEvent); 
    CFRelease(theEvent); 
} 
@end

You may need to play with the sleep which is the time interval between pushing the mouse button and releasing it. I have found that using 1 second sometimes it does not do it. Putting 2 seconds make it work all the time.
So your main would to the following.

int main(int argc, char *argv[])
{
    NSAutoreleasePool *pool    = [[NSAutoreleasePool alloc] init];
    NSUserDefaults *args    = [NSUserDefaults standardUserDefaults];

    // Grabs command line arguments -x, -y, -clicks, -button
    // and 1 for the click count and interval, 0 for button ie left.
    int x        = [args integerForKey:@"x"];
    int y        = [args integerForKey:@"y"];
    int clicks    =    [args integerForKey:@"clicks"];
    int button   =    [args integerForKey:@"button"];
    //int interval=    [args integerForKey:@"interval"];
    int resultcode;

   // PUT DEFAULT VALUES HERE WHEN SENT WITH EMPTY VALUES
    /*if (x==0) {
        x= 1728+66;
        y= 89+80;
        clicks=2;
        button=0;
    }
     */
    // The data structure CGPoint represents a point in a two-dimensional
    // coordinate system.  Here, X and Y distance from upper left, in pixels.
    CGPoint pt;
    pt.x    = x;
    pt.y    = y;



    // Check CGEventPostToPSN Posts a Quartz event into the event stream for a specific application.
    // only added the front lines plus  changed null in Create Events to kCGHIDEventTap

    gamePSN *gameData = [[gamePSN alloc] init];
    [gameData getPSN];
    ProcessSerialNumber psn = [gameData gamePSN]; 
    resultcode = SetFrontProcess(&psn);

    mouseClicks *mouseEvent =[[mouseClicks alloc] init];

    if (button == 0)
    {
        if (clicks==1) {
            [mouseEvent LeftClick:pt];
        } else {
            [mouseEvent doubleLeftClick:pt];
        }
    }   
    if (button == 1)
    {
        if (clicks==1) {
            [mouseEvent RightClick:pt];
        } else {
            [mouseEvent doubleRightClick:pt];
        }
    }   

    [gameData release];
    [mouseEvent release];

    [pool drain];
    return 0;
}

Hope this is helpful…. remember you can execute this in the terminal or in AppleScript by sending the following command.

do shell script "/...Path to Compiled Program.../ClickDep" & " -x " & someX & " -y " & someY & " -clicks 1 -button 1"

HAPPY GAMING!!!!