Continual Learning for Robotics: Definition, Framework, Learning
  Strategies, Opportunities and Challenges
  Continual learning (CL) is a particular machine learning paradigm where the
data distribution and learning objective changes through time, or where all the
training data and objective criteria are never available at once. The evolution
of the learning process is modeled by a sequence of learning experiences where
the goal is to be able to learn new skills all along the sequence without
forgetting what has been previously learned. Continual learning also aims at
the same time at optimizing the memory, the computation power and the speed
during the learning process.
  An important challenge for machine learning is not necessarily finding
solutions that work in the real world but rather finding stable algorithms that
can learn in real world. Hence, the ideal approach would be tackling the real
world in a embodied platform: an autonomous agent. Continual learning would
then be effective in an autonomous agent or robot, which would learn
autonomously through time about the external world, and incrementally develop a
set of complex skills and knowledge.
  Robotic agents have to learn to adapt and interact with their environment
using a continuous stream of observations. Some recent approaches aim at
tackling continual learning for robotics, but most recent papers on continual
learning only experiment approaches in simulation or with static datasets.
Unfortunately, the evaluation of those algorithms does not provide insights on
whether their solutions may help continual learning in the context of robotics.
This paper aims at reviewing the existing state of the art of continual
learning, summarizing existing benchmarks and metrics, and proposing a
framework for presenting and evaluating both robotics and non robotics
approaches in a way that makes transfer between both fields easier.
