Scaling MAP-Elites to Deep Neuroevolution
  Quality-Diversity (QD) algorithms, and MAP-Elites (ME) in particular, have
proven very useful for a broad range of applications including enabling real
robots to recover quickly from joint damage, solving strongly deceptive maze
tasks or evolving robot morphologies to discover new gaits. However, present
implementations of MAP-Elites and other QD algorithms seem to be limited to
low-dimensional controllers with far fewer parameters than modern deep neural
network models. In this paper, we propose to leverage the efficiency of
Evolution Strategies (ES) to scale MAP-Elites to high-dimensional controllers
parameterized by large neural networks. We design and evaluate a new hybrid
algorithm called MAP-Elites with Evolution Strategies (ME-ES) for post-damage
recovery in a difficult high-dimensional control task where traditional ME
fails. Additionally, we show that ME-ES performs efficient exploration, on par
with state-of-the-art exploration algorithms in high-dimensional control tasks
with strongly deceptive rewards.
