Deep Counterfactual Regret Minimization
  Counterfactual Regret Minimization (CFR) is the leading framework for solving
large imperfect-information games. It converges to an equilibrium by
iteratively traversing the game tree. In order to deal with extremely large
games, abstraction is typically applied before running CFR. The abstracted game
is solved with tabular CFR, and its solution is mapped back to the full game.
This process can be problematic because aspects of abstraction are often manual
and domain specific, abstraction algorithms may miss important strategic
nuances of the game, and there is a chicken-and-egg problem because determining
a good abstraction requires knowledge of the equilibrium of the game. This
paper introduces Deep Counterfactual Regret Minimization, a form of CFR that
obviates the need for abstraction by instead using deep neural networks to
approximate the behavior of CFR in the full game. We show that Deep CFR is
principled and achieves strong performance in large poker games. This is the
first non-tabular variant of CFR to be successful in large games.
