Artificial Intelligence is one of the most sought-after pieces of intelligence today. Businesses all over the world are seeking to use AI to revolutionize business analytics and lead to a better business strategy. Poker is no different than the rest of these businesses. Poker players all over are turning to AI to revolutionize their game, and poker AI is certainly as cutting edge as it gets.
Poker AI: The Beginning
Poker AI, or solvers as most players refer to them, are computer programs that simulate thousands of hands to determine the optimal strategy. These poker AI programs will simulate hands against themselves using game theory optimal decision-making to determine the best strategy with the given information. It is essentially poker AI playing poker against AI. These programs can analyze massive data sets and different situations in an instant.
According to researchers at Carnegie Mellon University, heads up No Limit Holdem has 10^161 different situations that can occur. That is a 1 with 161 zeros following it. A poker player can’t consider a fraction of those possibilities, but AI has the power to analyze far more than the human mind can.
The early versions of these AI poker bots came from universities. Professor Tuomas Sandholm and his graduate students at Carnegie Mellon University developed Claudico, the name comes from the Latin word for “I limp”. The AI poker bot learned how to play Texas Hold’em through statistical analysis, analyzing different possibilities and playing hands against itself to recognize patterns to find the optimal strategy. This is the basis for GTO solvers, studying the results of AI poker bot vs. AI poker bot. As development continued, Claudico competed against other university and freelance poker AI bots to see who had developed the most effective poker AI software. Then, Claudico was put to the test.
Professional Poker vs. AI Bot: Round 1
From April 24 to May 8, 2015, Claudico competed against four professional poker players, Doug Polk, Dong Kim, Jason Les, and Bjorn Li at the Rivers Casino in Pittsburgh. Each player would play 20,000 hands heads up against Claudico. To account for variance, while one player played on the main casino floor, another would be in a back room playing the same hands with the hole cards flipped. This way the results wouldn’t be skewed by a streak of run good.
Throughout the match, the poker players noted the bot doing some strange things. There were instances where it would bet incredibly small, or bet 20x the pot size. These are not typical of human poker players. As Doug Polk said “Betting $19,000 to win a $700 pot just isn’t something that a person would do,” but that didn’t stop Claudico. While the poker AI fought hard, the humans ended up victorious. Playing 50/100 blinds, the collective results had the humans ahead by 732,713 chips, roughly a 9.15BB/100 win rate.
Despite the poker pros coming out ahead, this was still an amazing feat for poker AI. Claudico was able to learn during the match and make adjustments based on their opponent. Professor Sandholm and his students went back to campus with their results and continued working to improve poker AI.
Poker AI Hits the Open Market
Shortly after Claudico’s match against the pros, solvers hit the open market for players to work on their game. One of the most notable, PioSolver, was released in March 2015 and it cemented itself as the premier poker AI. Professionals at all levels were turning to solvers to improve their game. The AI poker solver could quickly solve heads-up postflop situations, giving players the optimal strategy.
The basis for GTO and poker AI is to be unexploitable. You will often hear players say there are two strategies to poker, a GTO approach or an exploitable one. There is no argument, you will make more money if you can effectively exploit your opponent’s weaknesses.
For example, if you notice someone always bets large when they have a good hand and always bets small when they are bluffing, you can exploit that by raising their small bets and folding to their large bets. Another example is in a live $1/$2 game players typically open raise with the same sizing every hand. Some players will open to $6, others to $10, but typically if they are opening the pot (without limpers) they will use the same size with every hand. If you notice a player usually opens to $7, then all of a sudden opens to $15 and shows down AA, they may be using larger open sizes with premium hands. If they continue to do this throughout a session, you can adjust to 3-betting their smaller raises more and playing tight against their larger opens.
Poker AI gives you multiple options at different frequencies so you cannot be exploited like those previous examples. For instance, a solver may give you four options for a postflop spot like this: Check 35%, bet ¼ pot 45%, bet ¾ pot 15%, and bet full pot 5%. Players can look at this and see they should be betting a majority of the time, and the preferred sizing is ¼ pot. But to truly be unexploitable, you can’t always go with the preferred option. These frequencies are not used to find the best option and stick with it all the time, you should adhere to the percentages. With an outlier like the 5% full pot, some players will remove that strategy and add it to a closer, more common one.
Players will often use a “randomizer” to determine what they will do. If someone knows they should be checking 35%, betting small 45%, and betting big 20%, they may look at their watch. If the second hand is in the first 20 seconds they check, if it is in the last 10 seconds they bet big, anything in between they bet small. This is how poker players strive to be unexploitable, by using poker AI solvers to find the optimal frequencies and using some sort of randomizer to choose what they will do.
PioSolver and other poker AI software revolutionized the poker world. While studying used to be done by talking with other players in person or online in forums like 2+2, it could now be done using poker AI. Players don’t need to rely on other players for advice, or simply play over and over to recognize patterns and strategies themselves. Now, players can have the AI play itself and study the findings. The old-school poker pros had to get better by outthinking and out learning others over years of practice on the felt. Then online poker came and players could see the same amount of hands in one year that a live player could in 10, allowing for rapid improvement. With poker AI, players don’t even need to play to get better. They can use solvers to run simulations, study common scenarios, and use the computing power of AI to improve their game. AI has completely revolutionized how poker is studied and played.
While these AI poker bots were incredible, there are so many possibilities in each hand that the early versions could take a while to develop optimal solutions. It would take a significant amount of time to study and learn from these solutions, so no human player would ever be able to learn enough to play a completely unexploitable GTO theory. However, back at Carnegie Mellon professors and students were working on another poker AI bot, something that could analyze thousands of scenarios in the blink of an eye.
Professional Poker vs. AI Bot: Round 2
In 2017, the heads-up poker vs AI match was brought back, this time the bot challenging professionals was called Libratus. The poker pros brought in to take down poker AI were Jason Les, Dong Kim, Daniel McAulay, and Jimmy Chou. This time they elected to play 120,000 hands instead of 80,000. The format stayed the same where one player would be on the floor and another in a back room with the hole cards flipped to account for luck. Unlike its predecessor, Libratus would not taste defeat.
The poker bot took the professionals for over 1.7 million chips. With a win rate of 14.7BB/100, it was clear poker AI had come out on top. There were significant improvements to the software as Libratus was able to adapt much more effectively to its opponent’s strategies. While the players would relax and get some sleep after a match, Libratus would analyze the hands played, recognize patterns in the human’s play, and develop solutions to beat their strategy.
This is the significant advantage poker AI has over humans. Professionals have other things to deal with, family, sleep, exercise, entertainment, etc. We are humans after all, we can’t do one thing our entire lives. Poker AI, however, has one sole purpose: to play the best poker. With the brain power of millions (or billions) of humans, poker AI can learn more about the game in one day than most of us can in a lifetime. And unlike humans, the machine will not forget. The speed at which poker AI can develop optimal strategies does cause some concern in the poker community.
Real-Time Assistance with Poker AI
As poker AI advanced, many players were worried about poker bots playing online, and they were right. There have been multiple accounts flagged and banned from leading online sites because they aren’t players at all, but poker bots designed to play a near-perfect GTO strategy. The makers of these bots can create an account, load their bot, and watch it rake in pots. Security teams on these sites are adapting and getting much better at recognizing and banning bot accounts, but it is not perfect.
Additionally, some players have been accused of using poker AI to give them real-time assistance. Using a solver, or dream machine, that can compute the optimal strategy in real-time without a long delay would give players a significant edge. Poker AI this strong isn’t commercially available to the public. There are some sites with fast solvers, but you will find they use a database of pre-solved spots, so they cannot have the solution to everything. However, there may be some private poker AI bots available to a few that can do this. Poker AI is a wonderful tool that helps players get better, but as displayed by the grudge matches, AI has conquered poker players and can be a threat to online games.
While these are concerning, poker sites are constantly investing in security to remove any potential bots or players using RTA (real-time assistance). Overall, most games are safe. Poker AI bots are not easy to come by, they either require a lot of money or a high level of coding ability. Between the security and barrier to entry for poker bots, the majority of online games will not have one. However, if you are in a game with a bot, it will not be easy to win. While Libratus proved AI can defeat poker players heads up, another bot decided to show humans they can conquer more than heads up.
Professional Poker vs. AI Bot: Round 3
In 2019, another AI poker bot from Carnegie Mellon challenged professional poker players, only this time it wouldn’t be a heads-up match. Pluribus, the newest bot, challenged five professionals to play in a 6-handed NLH game. As noted, heads up NLH has an astronomical amount of possible situations, but 6-max has even more. With four more players in the mix, there are a seemingly endless amount of situations that can occur. In short, the more players in the game the harder it is to solve. Many assumed that AI wouldn’t be strong enough to solve a 6-max game in real-time. They were wrong.
Darren Elias helped test the bot by playing a 6-handed game with himself and five versions of the bot. The bot proved successful in this format. To test its strength further, Pluribus then played against five professionals at once. The poker AI bot crushed the field over 10,000 hands with a win rate of 5 chips per hand or 1000/hour.
This was the nail in the coffin. With decisive victories both in heads-up and 6-max play, it was clear AI conquered poker. Elias stated “There’s no going back. The bots will always be better than us.”
AI poker bots have proven their ability to effectively beat humans at the game. The computing power and ability to recognize optimal strategies are unmatched. While players may be concerned about online integrity, this revolution in poker is something to be celebrated. With the power of poker AI software, players can learn effective strategies much faster than before. There has never been more information about poker, and with the enhancements in AI, the information will continue to grow and expand, giving players everything they need to become unstoppable at the table.
