Question

There is a new Open Source poker bot called PokerPirate. I am interested in any creative ways in which a web application could detect/thwart/defeat a poker bot. (This is a purely academic discussion, in the same spirit that PokerPirate was written.)

Answer 1

Defeating a bot from the serverside perspective

1. Many online poker sites use popup
   Captcha inputs that are triggered by
   suspicious activity.

2. Some poker sites monitor playing
   times and patterns (i.e., worst case
   scenario is a player who plays 24×7
   and 16 tables continuously, there is
   a tiny tiny chance this is a real
   human. (However some players do have the ability to play very large hand volumes which to the inexperienced eye would appear to be a bot)

3. Throw it glitches. If you suspect a
   player is a bot, change all their
   playing card positions off a few
   pixels on the screen, make them
   different colours/designs/patterns
   for 1/100 hands and see if it throws
   them. If it can’t screen grab it
   will time-out on all its decisions
   and that’s pretty conclusive bot
   evidence.

4. Timing tells, if a computer player
   responds to options in milliseconds
   at a time without pause for thought
   on large decisions this could be
   suspicious

5. Self monitoring. The poker website
   pokertableratings.com data
   mines a lot of large sites. It has
   been met with a mixed reception,
   some love the transparency, others
   hate it. The benefit is, however,
   that there have been instances
   where suspicious player
   statistics (VPIP percentages,
   PFR percentages are a few of a large
   number of quantifiable statistics
   that can be recorded) have lead to
   conclusions of cheating

6. Artificially intelligent
   classification networks could
   monitor quantifiable statistics to
   classify rogue cheating or robotic
   players.

7. Back when online poker was a fairly
   new entity, there was rumour and talk with limited evidence that
   some poker client software
   screen-shots of suspicious players
   desktops to see if they were running
   programs that assist them. However (even if this were true) running two computers to perform the two tasks independently would get around this.

8. Sharing information between repeat
   offenders between multiple sites
   would be beneficial to the industry,
   if only they were honourable and run by competent responsible people

9. Some bots would probably be quite
   simple by design, if you could discover their
   playing style and see how they act
   in identical situations (note this
   is only possible with
   unsophisticated bots playing very
   basic strategy) you could discover
   them reasonably quickly.

10. Inconsistent use of program
    features would lean towards a
    player being genuine. Take for
    example many poker sites in game
    have a ‘Fold when it’s my turn’
    button. If you get dealt a bad
    hand and are waiting for another
    player to decide what to do, a lot
    of players will check this button.
    A bot may use these buttons. The
    difference is, a bot would be on the extremities of frequency of use, they would probably either use them all the time, or not at all.
    Wheras a player might usually press
    ‘autofold’, but sometimes they will
    click fold anyway even in the most
    favorable conditions. For example,
    a genuine player usually presses
    auto fold but this time they don’t.
    It’s folded round to them with no
    other player acting, now they have
    been presented with the most
    favourable condition possible. Now
    if they press fold, they would have
    been heavily inclined to press autofold from the start. This is
    inconsistant/unoptimised/random
    behaviour, consistant with being a
    human. Timing tells on when these features are clicked are other indicators. It is important to recognise that these are all indicators and not conclusive proof. All of these behavioural indicators can be simulated easily.

Defeating a bot from a players perspective

10. Try to log and collect as much data as possible using software like PokerTracker

11. Attempt to identify patterns in
    its playing style

12. Attempt to find relationships
    between bet size in proportion to
    pot/# players and hand strength

13. Try to calculate its hand ranges. A low stakes bot probably wont be bluffing frequently enough to be of any significant strategic concern, so constructing highly accurate hand ranges for it shouldn’t be too tricky.

14. Attempt to find leaks in its
    game via data analysis and trial
    and error Once leaks/patterns have
    been found, attempt to repetitively
    exploit them and avoid any other
    situations.

Where a human is capable of adaptation, bots probably are less so, and where humans are weighted by the chains of tilt, results orientated thinking and frustrations, bots are not. You can use this to your advantage.

So in essence there is nothing you can do to stop it if the robot is clever enough to simulate real timing delays during decisions, as well as create reasonable and realistic playing patterns. Throw in some random conditions and simple back-chat (the poker players lexicon is usually fairly limited) and you have yourself a AI player that’s going to be pretty hard to detect.

What bots might do to avoid detection

The key to avoid detection would be to think about the problem from as many angles as possible. You are attempting to simulate intelligent human behaviour in a very small and restricting world. Most of the behavioural simulations you can run are fairly obvious, but the more inconsistant and unpredictable your bot is, the less likely it is to be discovered.

1. Create realistic playing schedules
   (i.e., 3–5 times a week, 4 hours per
   session with the odd week here and
   there off during the year).

2. Run programs to make decisions on
   separate computer, controlling a
   zombie computer in case any sites
   screen capture.

3. Randomise action timings (don’t act
   immediately, wait 0.5–2 seconds per
   action)

4. Time down on big decisions. If a
   decision is borderline, calculate
   the decision then wait a while to
   simulate thought.

5. Random use of client software features.
   Simulate toilet breaks by clicking
   the “deal me out button” on all the
   tables and have a 5 minute break
   every now and then.

6. Simulated chat, poker chat is often
   very simple one liners, never
   usually discussion or debate. Say
   things like “unlucky” or “stfu” at
   appropriate detectable moments. Or
   even have the coder monitoring his
   bot and engaging in chat during
   execution.

7. Ensure mouse movements are
   realistic. If tables are
   tiled don’t make a decision on
   top left table then instantly make
   on on bottom right table. Most sites software now offer keyboard shortcuts, these may be preferable to use as supposed to mousemovement.

8. Do things that quite simply AI
   classifiers wont be expecting. For
   example, once a year phone them up
   with a simple non-complex query (“Help I can’t
   log in today!” or “The Internet is down!”) Unlikely to make much difference, but if the person working for the poker company is smart enough they might have recognised it as a realiable indicator.

9. Sporadic losing sessions. Tilt can
   be simulated and the bot can play
   badly and lose some money every now
   and then. Everybody tilts at some
   point.

The concern is also that poker websites don’t particularly care if bots are running on their networks, each player is worth a large amount in rake and theoretically from a purely cynical business point of view the only downside would be bad press if it was discovered.

Even when blatant exploits have been discovered, (search on google for Cereus network scandals or Absolute Poker Scandal, it’s quite shocking) the business appears to survive and remain healthy, only losing well educated and winning players (of which there are not many). This increases the proportion of less skilled players to the network, which in turn attracts the good players back. It’s a good ol’ fashioned catch 22. An excellent argument for proper market regulation.

It is important to note, that for every game a nash equilibrium exists. Online poker has a timeline to it the way it runs now, it’s going to have to move into something more social (webcam/voip) for anyone to trust it in the future (if people trust it) as bots will take over eventually as mathematically superior, and psychologically immune. The poker AI community is very active, fuelled by academia and/or capital benefit.

Simpler versions of poker such as limit poker have been very nearly solved in small search spaces. It’s only a matter of time before more complex versions of the game (No Limit variations/Pot Limit Omaha etc) become beatable for artificial players.

Conclusion

Sophisticated bots just can’t be detected until the industry shifts to a more social online gaming setting. This won’t solve the problem, but will certainly make it harder for bots to win at the lower levels. We’ve already seen a slight shift with the release of PKR, 3D and a more interactive, less hands per hour version of the other sites where multitabling is quite tricky to accomplish for a player.

The problem also suffers from the nature of the industry, yet another reason to stick to the larger more reputable websites where reputation has become more and more integrated into their business model. Lack of transparency and feigned transarancy don’t help the cause.

The real challenge currently for bot developers is to write a winning algorithm, this is not as trivial as it seems. Everyone who plays poker considers themselves good, winning or a break even player, which is simply not true. That is why people continue to play, even when they lose money as they are under the illusion they are simply unlucky, or their style of play is misunderstood. This arrogance and weakness in human psychology has cost losing players a lot of money and is the fundamental reason that poker can still be profitable.

Poker is a vastly complicated game that takes years to get good at (The old adage remains true, “Ten minutes to learn, a lifetime to master”). The luck element is extremely limited in the long term.

Like any other profession, to get good, you need to study for hundreds upon hundreds of hours, and play for many thousands. You will understand things that less experienced players wont understand, spot things the less experienced wont spot. The learning goes on for a very very long time, perhaps longer than we can ever live. It’s a complicated game.

How often have you seen a high stakes cash game on the television and heard someone shout at it “That’s an easy call!” thus prooving that amateurs really don’t understand or recognise sophistication in play, and truly beleive the game at that level is still ultimately simple. It isn’t. Those high stakes players (a lot of the time) are there on the television because they are really really really good. There is also probably a complicated meta game being played as well, which our amauer can’t recognise the existance of. The amatuer wouldn’t stand over a chess master and shout at them to move their knight, yet because of the dynamic of poker being imperfect information their psychology makes them truly beleive what they are saying. Like in chess, decisions can be intricate, sensitive and extremely important to the overall game. As the game increases in complexity, trivial decisions are not so trivial anymore, because your opponent expects them.

Once you move your bot or your game up the levels, you inevitably will come across a larger populous of more skilled players. Then, the complexity of your strategy is going to have to go up to the next level, taking into account table images, range balancing, sophisticated and intelligent bluffing (IE not just bluffing at weakness, bluffing at ranges and bluffing on image etc), with more detailed hand range analysis. It really is a different game as you move up.

Once a winning bot has been written, without doubt the coder will have enough skill, knowledge and common sense to apply the bot in an undetectable fashion. This is trivial for them.

So there really is nothing you can do. If you want to play online, understand the risks. Never risk more money than you can afford, and attempt to keep accurate records of spending so you don’t have a misguided, unrealistic and ultimately damaging over estimation of your own ability. Have stop losses, and leave the table if you don’t have an edge, or if you are unsure if you have an edge! Of course, if everyone did that no one would win, that’s the predatory and exploitative nature of the game, that’s where the competition comes from and that’s what makes it fun.