Meta Automatic Curriculum Learning
  A major challenge in the Deep RL (DRL) community is to train agents able to
generalize their control policy over situations never seen in training.
Training on diverse tasks has been identified as a key ingredient for good
generalization, which pushed researchers towards using rich procedural task
generation systems controlled through complex continuous parameter spaces. In
such complex task spaces, it is essential to rely on some form of Automatic
Curriculum Learning (ACL) to adapt the task sampling distribution to a given
learning agent, instead of randomly sampling tasks, as many could end up being
either trivial or unfeasible. Since it is hard to get prior knowledge on such
task spaces, many ACL algorithms explore the task space to detect progress
niches over time, a costly tabula-rasa process that needs to be performed for
each new learning agents, although they might have similarities in their
capabilities profiles. To address this limitation, we introduce the concept of
Meta-ACL, and formalize it in the context of black-box RL learners, i.e.
algorithms seeking to generalize curriculum generation to an (unknown)
distribution of learners. In this work, we present AGAIN, a first instantiation
of Meta-ACL, and showcase its benefits for curriculum generation over classical
ACL in multiple simulated environments including procedurally generated parkour
environments with learners of varying morphologies. Videos and code are
available at https://sites.google.com/view/meta-acl .
