Robust Reinforcement Learning via Genetic Curriculum
  Achieving robust performance is crucial when applying deep reinforcement
learning (RL) in safety critical systems. Some of the state of the art
approaches try to address the problem with adversarial agents, but these agents
often require expert supervision to fine tune and prevent the adversary from
becoming too challenging to the trainee agent. While other approaches involve
automatically adjusting environment setups during training, they have been
limited to simple environments where low-dimensional encodings can be used.
Inspired by these approaches, we propose genetic curriculum, an algorithm that
automatically identifies scenarios in which the agent currently fails and
generates an associated curriculum to help the agent learn to solve the
scenarios and acquire more robust behaviors. As a non-parametric optimizer, our
approach uses a raw, non-fixed encoding of scenarios, reducing the need for
expert supervision and allowing our algorithm to adapt to the changing
performance of the agent. Our empirical studies show improvement in robustness
over the existing state of the art algorithms, providing training curricula
that result in agents being 2 - 8x times less likely to fail without
sacrificing cumulative reward. We include an ablation study and share insights
on why our algorithm outperforms prior approaches.
