Monte-Carlo Graph Search for AlphaZero
  The AlphaZero algorithm has been successfully applied in a range of discrete
domains, most notably board games. It utilizes a neural network, that learns a
value and policy function to guide the exploration in a Monte-Carlo Tree
Search. Although many search improvements have been proposed for Monte-Carlo
Tree Search in the past, most of them refer to an older variant of the Upper
Confidence bounds for Trees algorithm that does not use a policy for planning.
We introduce a new, improved search algorithm for AlphaZero which generalizes
the search tree to a directed acyclic graph. This enables information flow
across different subtrees and greatly reduces memory consumption. Along with
Monte-Carlo Graph Search, we propose a number of further extensions, such as
the inclusion of Epsilon-greedy exploration, a revised terminal solver and the
integration of domain knowledge as constraints. In our evaluations, we use the
CrazyAra engine on chess and crazyhouse as examples to show that these changes
bring significant improvements to AlphaZero.
