Automatic tuning of hyper-parameters of reinforcement learning
  algorithms using Bayesian optimization with behavioral cloning
  Optimal setting of several hyper-parameters in machine learning algorithms is
key to make the most of available data. To this aim, several methods such as
evolutionary strategies, random search, Bayesian optimization and heuristic
rules of thumb have been proposed. In reinforcement learning (RL), the
information content of data gathered by the learning agent while interacting
with its environment is heavily dependent on the setting of many
hyper-parameters. Therefore, the user of an RL algorithm has to rely on
search-based optimization methods, such as grid search or the Nelder-Mead
simplex algorithm, that are very inefficient for most RL tasks, slows down
significantly the learning curve and leaves to the user the burden of
purposefully biasing data gathering. In this work, in order to make an RL
algorithm more user-independent, a novel approach for autonomous
hyper-parameter setting using Bayesian optimization is proposed. Data from past
episodes and different hyper-parameter values are used at a meta-learning level
by performing behavioral cloning which helps improving the effectiveness in
maximizing a reinforcement learning variant of an acquisition function. Also,
by tightly integrating Bayesian optimization in a reinforcement learning agent
design, the number of state transitions needed to converge to the optimal
policy for a given task is reduced. Computational experiments reveal promising
results compared to other manual tweaking and optimization-based approaches
which highlights the benefits of changing the algorithm hyper-parameters to
increase the information content of generated data.
