Model-Based Quality-Diversity Search for Efficient Robot Learning
  Despite recent progress in robot learning, it still remains a challenge to
program a robot to deal with open-ended object manipulation tasks. One approach
that was recently used to autonomously generate a repertoire of diverse skills
is a novelty based Quality-Diversity~(QD) algorithm. However, as most
evolutionary algorithms, QD suffers from sample-inefficiency and, thus, it is
challenging to apply it in real-world scenarios. This paper tackles this
problem by integrating a neural network that predicts the behavior of the
perturbed parameters into a novelty based QD algorithm. In the proposed
Model-based Quality-Diversity search (M-QD), the network is trained
concurrently to the repertoire and is used to avoid executing unpromising
actions in the novelty search process. Furthermore, it is used to adapt the
skills of the final repertoire in order to generalize the skills to different
scenarios. Our experiments show that enhancing a QD algorithm with such a
forward model improves the sample-efficiency and performance of the
evolutionary process and the skill adaptation.
