Towards Automatic Actor-Critic Solutions to Continuous Control
  Model-free off-policy actor-critic methods are an efficient solution to
complex continuous control tasks. However, these algorithms rely on a number of
design tricks and hyperparameters, making their application to new domains
difficult and computationally expensive. This paper creates an evolutionary
approach that automatically tunes these design decisions and eliminates the
RL-specific hyperparameters from the Soft Actor-Critic algorithm. Our design is
sample efficient and provides practical advantages over baseline approaches,
including improved exploration, generalization over multiple control
frequencies, and a robust ensemble of high-performance policies. Empirically,
we show that our agent outperforms well-tuned hyperparameter settings in
popular benchmarks from the DeepMind Control Suite. We then apply it to less
common control tasks outside of simulated robotics to find high-performance
solutions with minimal compute and research effort.
