Static and Dynamic Values of Computation in MCTS
  Monte-Carlo Tree Search (MCTS) is one of the most-widely used methods for
planning, and has powered many recent advances in artificial intelligence. In
MCTS, one typically performs computations (i.e., simulations) to collect
statistics about the possible future consequences of actions, and then chooses
accordingly. Many popular MCTS methods such as UCT and its variants decide
which computations to perform by trading-off exploration and exploitation. In
this work, we take a more direct approach, and explicitly quantify the value of
a computation based on its expected impact on the quality of the action
eventually chosen. Our approach goes beyond the "myopic" limitations of
existing computation-value-based methods in two senses: (I) we are able to
account for the impact of non-immediate (ie, future) computations (II) on
non-immediate actions. We show that policies that greedily optimize computation
values are optimal under certain assumptions and obtain results that are
competitive with the state-of-the-art.
