Improving Generalization in Meta Reinforcement Learning using Learned
  Objectives
  Biological evolution has distilled the experiences of many learners into the
general learning algorithms of humans. Our novel meta reinforcement learning
algorithm MetaGenRL is inspired by this process. MetaGenRL distills the
experiences of many complex agents to meta-learn a low-complexity neural
objective function that decides how future individuals will learn. Unlike
recent meta-RL algorithms, MetaGenRL can generalize to new environments that
are entirely different from those used for meta-training. In some cases, it
even outperforms human-engineered RL algorithms. MetaGenRL uses off-policy
second-order gradients during meta-training that greatly increase its sample
efficiency.
