How AI Mastered Poker Using Regret Minimization

Artificial intelligence achieved a historic milestone when Libratus and Pluribus defeated elite human professionals at No-Limit Texas Hold'em. Unlike chess or Go, poker is an imperfect-information game where players cannot see their opponents' cards, requiring the AI to navigate hidden information, bluffing, and unpredictable human behavior. Both AI systems relied on an advanced algorithm called Counterfactual Regret Minimization (CFR) to calculate unexploitable strategies. By combining CFR with game abstractions and real-time search techniques, Libratus conquered two-player poker, and Pluribus later mastered six-player games, cementing AI superiority in complex strategic decision-making.

The Challenge of Imperfect Information

In perfect-information games like chess, both players see the entire board. Poker is an imperfect-information game because cards are hidden, and chance plays a role. Furthermore, No-Limit Texas Hold'em allows players to bet any amount of chips at any time, creating a massive number of possible decision points—exceeding the number of atoms in the universe. To win, an AI cannot simply predict the opponent's best move; it must balance its own strategy to remain unpredictable, including learning how and when to bluff.

Counterfactual Regret Minimization (CFR) Explained

The core engine behind both Libratus and Pluribus is Counterfactual Regret Minimization (CFR). CFR is an iterative self-play algorithm where the AI plays billions of hands against itself to learn from its mistakes.

At the end of a simulated hand, the algorithm looks back at every decision it made and calculates "regret." Regret is a numerical value representing how much better the AI would have performed if it had chosen a different action (such as folding instead of calling, or raising instead of checking).

In future games, the AI adjusts its strategy to favor actions that yielded higher positive regret in the past. Over trillions of iterations, the average of these strategies converges toward a Nash Equilibrium—a state in game theory where no player can gain an advantage by unilaterally changing their strategy. Even if a human knows the AI's exact strategy, they cannot exploit it in the long run.

Game Abstraction: Simplifying the Math

Because the game tree of No-Limit Texas Hold'em is too large to solve completely, the researchers used a technique called abstraction to shrink the game into a manageable size before running CFR:

  • Card Abstraction: Grouping similar hands together. For example, holding a King-High flush is treated practically the same as holding a Queen-High flush in many situations.
  • Action Abstraction: Limiting the number of bet sizes the AI considers. Instead of evaluating a bet of $101, $102, and $103 separately, the AI might only consider betting half the pot, the full pot, or going all-in.

This creates a simplified "blueprint" strategy that guides the AI through the early stages of a hand.

Libratus: Conquering Heads-Up Poker

Developed in 2017 by Tuomas Sandholm and Noam Brown at Carnegie Mellon University, Libratus was designed for two-player (heads-up) No-Limit Texas Hold'em. It defeated four top human specialists over 120,000 hands using a three-part system coupled with CFR:

  1. The Blueprint Strategy: Libratus used a supercomputer to run CFR on an abstracted version of the game, creating a baseline strategy.
  2. Nested Safe Subgame Solving: When a human made a bet size not included in the AI's abstraction (e.g., an unusual overbet), Libratus did not round the bet to the nearest abstracted value. Instead, it paused and calculated a brand-new, detailed strategy for that specific situation in real-time using CFR.
  3. Self-Improvement Module: Every night during the tournament, Libratus analyzed the bet sizes humans used most frequently to exploit holes in its blueprint. It then used supercomputing power overnight to fill those gaps before the next day's match.

Pluribus: Scaling to Six Players

In 2019, Brown and Sandholm collaborated with Facebook AI to create Pluribus, tackling the much harder challenge of six-player poker. With more players, the computational complexity skyrockets, and converging on a true Nash Equilibrium becomes practically impossible.

To succeed, Pluribus modified the Libratus approach to be vastly more efficient, running on just two server nodes rather than a massive supercomputer:

  • Monte Carlo CFR: Pluribus used a faster variant of CFR that samples only certain branches of the game tree rather than traversing every possible outcome, allowing it to build a strong blueprint quickly.
  • Depth-Limited Search: In chess, AI looks ahead a few moves and estimates the value of the board. In poker, this is difficult because the value of a hand depends on what your opponents think you have. Pluribus solved this by looking only a few moves ahead and assuming that beyond that point, opponents might adopt one of four different strategies (such as playing aggressively or passively). It then used CFR to find a balanced response to all of those potential strategies simultaneously.
  • Deliberate Unpredictability: To prevent humans from reading its patterns, Pluribus randomized its actions in real-time, frequently mixing bluffs and value bets with the exact same bet sizes.

Through the combination of Counterfactual Regret Minimization, abstraction, and real-time depth-limited search, Libratus and Pluribus proved that AI can master environments defined by hidden information and strategic deception.