How computers solved poker

8 steps · read as text

strategies

equilibrium

bluffs, as a share of bets →↑ calls

round 0average: bluffs 49% · calls 10%

regret

bettor, with a miss

bluff 95%give up 5%

caller, facing a bet

call 10%fold 90%

milestones

2015CepheusHeads-up limit hold’em, essentially solved.

2017LibratusBeats top professionals at heads-up no-limit.

2019PluribusBeats professionals at six-player no-limit.

1 / 8

The strongest poker programs were never taught strategy. They play against themselves, and keep score of their regrets.

the method

regret(action) += what it would have won − what you won

play each action in proportion to its positive regret

The strategies keep moving. Their average converges to an equilibrium in two-player zero-sum games.

what players learned

Pluribus almost never limped, used large overbets, and led into the raiser more often than professionals do. Its edge came from mixing actions at the right frequencies, every time.

07Nash equilibrium 10ICM →Practice: push or fold

sources & further reading

A Simple Adaptive Procedure Leading to Correlated EquilibriumSergiu Hart, Andreu Mas-Colell · Econometrica · 2000 · paper

Regret Minimization in Games with Incomplete InformationZinkevich, Johanson, Bowling, Piccione · NeurIPS · 2007 · paper

Heads-up limit hold’em poker is solved ↗Bowling, Burch, Johanson, Tammelin · Science · 2015 · paper

Superhuman AI for heads-up no-limit poker ↗Noam Brown, Tuomas Sandholm · Science · 2018 · paper

Superhuman AI for multiplayer poker ↗Noam Brown, Tuomas Sandholm · Science · 2019 · paper