Planning and Learning Using Adaptive Entropy Tree Search
  We present the Adaptive Entropy Tree Search (ANTS) algorithm, a planning
method based on the Principle of Maximum Entropy. Importantly, we design ANTS
so that it is a practical component of a planning-learning loop, outperforming
state-of-the-art methods on the Atari benchmark. The key algorithmic novelty is
entropy parameterization, which mitigates sensitivity to the temperature
parameter - a bottleneck of the prior maximum entropy planning methods. To
confirm our design choices, we perform a comprehensive suite of ablations in
isolation from learning. Moreover, we theoretically show that ANTS enjoys
exponential convergence in the softmax bandit setting.
