Learning Transferable Motor Skills with Hierarchical Latent Mixture
  Policies
  For robots operating in the real world, it is desirable to learn reusable
behaviours that can effectively be transferred and adapted to numerous tasks
and scenarios. We propose an approach to learn abstract motor skills from data
using a hierarchical mixture latent variable model. In contrast to existing
work, our method exploits a three-level hierarchy of both discrete and
continuous latent variables, to capture a set of high-level behaviours while
allowing for variance in how they are executed. We demonstrate in manipulation
domains that the method can effectively cluster offline data into distinct,
executable behaviours, while retaining the flexibility of a continuous latent
variable model. The resulting skills can be transferred and fine-tuned on new
tasks, unseen objects, and from state to vision-based policies, yielding better
sample efficiency and asymptotic performance compared to existing skill- and
imitation-based methods. We further analyse how and when the skills are most
beneficial: they encourage directed exploration to cover large regions of the
state space relevant to the task, making them most effective in challenging
sparse-reward settings.
