Fast and stable MAP-Elites in noisy domains using deep grids
  Quality-Diversity optimisation algorithms enable the evolution of collections
of both high-performing and diverse solutions. These collections offer the
possibility to quickly adapt and switch from one solution to another in case it
is not working as expected. It therefore finds many applications in real-world
domain problems such as robotic control. However, QD algorithms, like most
optimisation algorithms, are very sensitive to uncertainty on the fitness
function, but also on the behavioural descriptors. Yet, such uncertainties are
frequent in real-world applications. Few works have explored this issue in the
specific case of QD algorithms, and inspired by the literature in Evolutionary
Computation, mainly focus on using sampling to approximate the "true" value of
the performances of a solution. However, sampling approaches require a high
number of evaluations, which in many applications such as robotics, can quickly
become impractical. In this work, we propose Deep-Grid MAP-Elites, a variant of
the MAP-Elites algorithm that uses an archive of similar previously encountered
solutions to approximate the performance of a solution. We compare our approach
to previously explored ones on three noisy tasks: a standard optimisation task,
the control of a redundant arm and a simulated Hexapod robot. The experimental
results show that this simple approach is significantly more resilient to noise
on the behavioural descriptors, while achieving competitive performances in
terms of fitness optimisation, and being more sample-efficient than other
existing approaches.
