Model-free Neural Counterfactual Regret Minimization with Bootstrap
  Learning
  Counterfactual Regret Minimization (CFR) has achieved many fascinating
results in solving large-scale Imperfect Information Games (IIGs). Neural
network approximation CFR (neural CFR) is one of the promising techniques that
can reduce computation and memory consumption by generalizing decision
information between similar states. Current neural CFR algorithms have to
approximate cumulative regrets. However, efficient and accurate approximation
in a large-scale IIG is still a tough challenge. In this paper, a new CFR
variant, Recursive CFR (ReCFR), is proposed. In ReCFR, Recursive Substitute
Values (RSVs) are learned and used to replace cumulative regrets. It is proven
that ReCFR can converge to a Nash equilibrium at a rate of $O({1}/{\sqrt{T}})$.
Based on ReCFR, a new model-free neural CFR with bootstrap learning, Neural
ReCFR-B, is proposed. Due to the recursive and non-cumulative nature of RSVs,
Neural ReCFR-B has lower-variance training targets than other neural CFRs.
Experimental results show that Neural ReCFR-B is competitive with the
state-of-the-art neural CFR algorithms at a much lower training cost.
