On the Evolution of the MCTS Upper Confidence Bounds for Trees by Means
  of Evolutionary Algorithms in the Game of Carcassonne
  Monte Carlo Tree Search (MCTS) is a sampling best-first method to search for
optimal decisions. The MCTS's popularity is based on its extraordinary results
in the challenging two-player based game Go, a game considered much harder than
Chess and that until very recently was considered infeasible for Artificial
Intelligence methods. The success of MCTS depends heavily on how the tree is
built and the selection process plays a fundamental role in this. One
particular selection mechanism that has proved to be reliable is based on the
Upper Confidence Bounds for Trees, commonly referred as UCT. The UCT attempts
to nicely balance exploration and exploitation by considering the values stored
in the statistical tree of the MCTS. However, some tuning of the MCTS UCT is
necessary for this to work well. In this work, we use Evolutionary Algorithms
(EAs) to evolve mathematical expressions with the goal to substitute the UCT
mathematical expression. We compare our proposed approach, called Evolution
Strategy in MCTS (ES-MCTS) against five variants of the MCTS UCT, three
variants of the star-minimax family of algorithms as well as a random
controller in the Game of Carcassonne. We also use a variant of our proposed
EA-based controller, dubbed ES partially integrated in MCTS. We show how the
ES-MCTS controller, is able to outperform all these 10 intelligent controllers,
including robust MCTS UCT controllers.
