Unlocking the Potential of Deep Counterfactual Value Networks
  Deep counterfactual value networks combined with continual resolving provide
a way to conduct depth-limited search in imperfect-information games. However,
since their introduction in the DeepStack poker AI, deep counterfactual value
networks have not seen widespread adoption. In this paper we introduce several
improvements to deep counterfactual value networks, as well as counterfactual
regret minimization, and analyze the effects of each change. We combined these
improvements to create the poker AI Supremus. We show that while a
reimplementation of DeepStack loses head-to-head against the strong benchmark
agent Slumbot, Supremus successfully beats Slumbot by an extremely large margin
and also achieves a lower exploitability than DeepStack against a local best
response. Together, these results show that with our key improvements, deep
counterfactual value networks can achieve state-of-the-art performance.
