A Simple Approach to Continual Learning by Transferring Skill Parameters
  In order to be effective general purpose machines in real world environments,
robots not only will need to adapt their existing manipulation skills to new
circumstances, they will need to acquire entirely new skills on-the-fly. A
great promise of continual learning is to endow robots with this ability, by
using their accumulated knowledge and experience from prior skills. We take a
fresh look at this problem, by considering a setting in which the robot is
limited to storing that knowledge and experience only in the form of learned
skill policies. We show that storing skill policies, careful pre-training, and
appropriately choosing when to transfer those skill policies is sufficient to
build a continual learner in the context of robotic manipulation. We analyze
which conditions are needed to transfer skills in the challenging Meta-World
simulation benchmark. Using this analysis, we introduce a pair-wise metric
relating skills that allows us to predict the effectiveness of skill transfer
between tasks, and use it to reduce the problem of continual learning to
curriculum selection. Given an appropriate curriculum, we show how to
continually acquire robotic manipulation skills without forgetting, and using
far fewer samples than needed to train them from scratch.
