Hyperparameter Tuning for Deep Reinforcement Learning Applications
  Reinforcement learning (RL) applications, where an agent can simply learn
optimal behaviors by interacting with the environment, are quickly gaining
tremendous success in a wide variety of applications from controlling simple
pendulums to complex data centers. However, setting the right hyperparameters
can have a huge impact on the deployed solution performance and reliability in
the inference models, produced via RL, used for decision-making. Hyperparameter
search itself is a laborious process that requires many iterations and
computationally expensive to find the best settings that produce the best
neural network architectures. In comparison to other neural network
architectures, deep RL has not witnessed much hyperparameter tuning, due to its
algorithm complexity and simulation platforms needed. In this paper, we propose
a distributed variable-length genetic algorithm framework to systematically
tune hyperparameters for various RL applications, improving training time and
robustness of the architecture, via evolution. We demonstrate the scalability
of our approach on many RL problems (from simple gyms to complex applications)
and compared with Bayesian approach. Our results show that with more
generations, optimal solutions that require fewer training episodes and are
computationally cheap while being more robust for deployment. Our results are
imperative to advance deep reinforcement learning controllers for real-world
problems.
