Learning to Stop: Dynamic Simulation Monte-Carlo Tree Search
  Monte Carlo tree search (MCTS) has achieved state-of-the-art results in many
domains such as Go and Atari games when combining with deep neural networks
(DNNs). When more simulations are executed, MCTS can achieve higher performance
but also requires enormous amounts of CPU and GPU resources. However, not all
states require a long searching time to identify the best action that the agent
can find. For example, in 19x19 Go and NoGo, we found that for more than half
of the states, the best action predicted by DNN remains unchanged even after
searching 2 minutes. This implies that a significant amount of resources can be
saved if we are able to stop the searching earlier when we are confident with
the current searching result. In this paper, we propose to achieve this goal by
predicting the uncertainty of the current searching status and use the result
to decide whether we should stop searching. With our algorithm, called Dynamic
Simulation MCTS (DS-MCTS), we can speed up a NoGo agent trained by AlphaZero
2.5 times faster while maintaining a similar winning rate. Also, under the same
average simulation count, our method can achieve a 61% winning rate against the
original program.
