♠ What poker researchers actually learned from AlphaZero-style self-play

In December 2017, DeepMind published a paper describing AlphaZero, a program that had taught itself chess, shogi, and Go using nothing but the rules of each game and self-play against its own earlier versions. Within hours it was outplaying programs built on decades of accumulated human opening theory. It became one of the defining AI stories of the decade: discard the textbooks, let the machine play itself into mastery.

Poker was running its own parallel experiment at almost the same moment, and the differences between the two stories tell us more than the similarities do.

We think the real lesson from this period is not that poker got "solved" the way chess appeared to be. It is that self-play, once researchers tried to apply it to a card game, ran straight into a structural fact that AlphaZero never had to face: in poker, nobody can see the whole board.

What self-play meant in a fully visible game

AlphaZero's method has real ancestry. Gerald Tesauro's TD-Gammon, built at IBM in the early 1990s, had already shown that a program could improve at backgammon by playing itself repeatedly and adjusting a value estimate after each game. AlphaZero scaled that idea enormously: a neural network proposes moves and estimates positions, a search procedure (Monte Carlo Tree Search) looks a few moves ahead using that network's judgment, and the results of self-play games are fed back to retrain the network, over and over, from random initialization.

This works cleanly in chess, shogi, and Go because both players always see the same board. There is, in principle, one objectively best move in any given position, independent of what either player believes about the other. Self-play against a strong enough version of yourself is a reasonable way to converge toward that move.

Why poker refuses that trick

Poker breaks this assumption at its foundation. Your hole cards are hidden from your opponent, and theirs are hidden from you. There is no single best action for a given hand, because the correct action depends on what your opponent believes you might be holding, and that belief depends in turn on how you have played in the past. The reasoning is circular, and it cannot be resolved by looking one level deeper the way chess search can. It has to be resolved by game theory.

This is why the poker research lineage that predates and runs alongside AlphaZero relies on counterfactual regret minimization, a self-play technique in its own right, but one built to converge toward a Nash equilibrium rather than toward "the correct move." A program plays millions of hands against earlier versions of itself, tracks how much it would have gained by having played differently in each situation, and slowly adjusts its strategy to reduce that regret across the board. What it converges to is not a fixed line of play but a distribution over actions, a strategy that cannot be reliably exploited no matter what the opponent does.

A poker program does not learn to read your hand; it learns to be unreadable, whatever hand it happens to hold.

Libratus, Pluribus, and the proof at the table

The clearest public demonstration came from Carnegie Mellon, where Noam Brown and Tuomas Sandholm built Libratus, a program that in January 2017 played a twenty-day heads-up no-limit hold 'em marathon against four elite professionals — Dong Kim, Jason Les, Daniel McAulay, and Jimmy Chou — at Rivers Casino in Pittsburgh. The humans shared a fixed prize pool regardless of the outcome, a deliberate design choice meant to keep them playing their honest best rather than defensively protecting a stake. Libratus won decisively, a reversal from Claudico, an earlier Carnegie Mellon program that had lost a similar match against professionals two years before.

In 2019, the same team built Pluribus and pushed the idea further, into six-player no-limit hold 'em. This mattered because the theoretical guarantees behind Nash equilibrium mostly apply cleanly to two-player zero-sum games; with six players at the table, those guarantees weaken or disappear. Many researchers expected multiplayer poker to resist this approach. Pluribus nonetheless produced a strategy, built through self-play and abstraction rather than any specific model of its human opponents, that outperformed a rotating field of professionals over thousands of hands.

• 2015 — Claudico (Carnegie Mellon) loses a heads-up match to professionals.
• 2017 — Libratus (Carnegie Mellon, Brown and Sandholm) defeats four heads-up specialists at Rivers Casino, Pittsburgh.
• 2017 — DeepStack (University of Alberta, Michael Bowling's group) introduces neural network value estimates inside a poker solver.
• 2019 — Pluribus (Carnegie Mellon / Facebook AI) beats professionals in six-player no-limit hold 'em.
• 2020 — ReBeL (Facebook AI Research) generalizes AlphaZero's self-play-and-search recipe to games of hidden information.

DeepStack and the return of the network

Around the same time, a University of Alberta group led by Michael Bowling released DeepStack, which replaced hand-built heuristics with a neural network trained through self-play to estimate the value of unresolved parts of the game tree. It was not yet AlphaZero's exact recipe, since the underlying search remained rooted in counterfactual regret rather than Monte Carlo Tree Search, but it moved deep learning back into the center of poker research, and it made the eventual convergence with AlphaZero-style methods look less like a coincidence and more like an inevitability.

ReBeL closes the loop

That convergence arrived in 2020, when Noam Brown and colleagues at Facebook AI Research published ReBeL, a method that explicitly generalizes AlphaZero's self-play-plus-search approach to imperfect-information games. The trick was to stop representing the state of the game as a single position, the way chess does, and instead represent it as a public belief state: a probability distribution, visible to everyone, over what each player might be holding given everything that has happened publicly so far. Once the state is redefined this way, a single AlphaZero-like algorithm can search and learn its way through poker, or through other hidden-information games, without game-specific engineering bolted on.

This is the real bridge between the two stories. It shows that AlphaZero's core idea — learn a value function through self-play, then use it to guide search — was never fundamentally restricted to perfect information. It simply needed a different definition of what a "position" is.

What this taught, and what it did not

The broader lesson for artificial intelligence research is that self-play is a genuinely powerful engine for discovering strategy without human data, but the shape of the problem still matters enormously. An assumption baked comfortably into one algorithm can be an invisible wall for another, and getting past that wall required an actual conceptual step, not just more computing power thrown at the same recipe.

For poker specifically, this body of research confirmed something thoughtful players had already sensed from game theory long before any of these programs existed: sound play is not about reading one hand correctly, it is about remaining unpredictable across every hand you could be holding. That is a finding about decision-making under uncertainty, with real echoes in auctions and negotiation research. It says nothing about turning poker into a source of income, and we would be doing our readers a disservice to imply otherwise.

What we find worth sitting with is the change in poker's academic standing that this decade produced. A game long treated as a matter of nerve and intuition became one of the sharpest available test cases for how machines reason under hidden information, standing alongside chess and Go rather than beneath them, and that shift in status is, on its own terms, a piece of poker history worth remembering.