Learning to Generalize Across Long-Horizon Tasks from Human
  Demonstrations
  Imitation learning is an effective and safe technique to train robot policies
in the real world because it does not depend on an expensive random exploration
process. However, due to the lack of exploration, learning policies that
generalize beyond the demonstrated behaviors is still an open challenge. We
present a novel imitation learning framework to enable robots to 1) learn
complex real world manipulation tasks efficiently from a small number of human
demonstrations, and 2) synthesize new behaviors not contained in the collected
demonstrations. Our key insight is that multi-task domains often present a
latent structure, where demonstrated trajectories for different tasks intersect
at common regions of the state space. We present Generalization Through
Imitation (GTI), a two-stage offline imitation learning algorithm that exploits
this intersecting structure to train goal-directed policies that generalize to
unseen start and goal state combinations. In the first stage of GTI, we train a
stochastic policy that leverages trajectory intersections to have the capacity
to compose behaviors from different demonstration trajectories together. In the
second stage of GTI, we collect a small set of rollouts from the unconditioned
stochastic policy of the first stage, and train a goal-directed agent to
generalize to novel start and goal configurations. We validate GTI in both
simulated domains and a challenging long-horizon robotic manipulation domain in
the real world. Additional results and videos are available at
https://sites.google.com/view/gti2020/ .
