AVID: Learning Multi-Stage Tasks via Pixel-Level Translation of Human
  Videos
  Robotic reinforcement learning (RL) holds the promise of enabling robots to
learn complex behaviors through experience. However, realizing this promise for
long-horizon tasks in the real world requires mechanisms to reduce human burden
in terms of defining the task and scaffolding the learning process. In this
paper, we study how these challenges can be alleviated with an automated
robotic learning framework, in which multi-stage tasks are defined simply by
providing videos of a human demonstrator and then learned autonomously by the
robot from raw image observations. A central challenge in imitating human
videos is the difference in appearance between the human and robot, which
typically requires manual correspondence. We instead take an automated approach
and perform pixel-level image translation via CycleGAN to convert the human
demonstration into a video of a robot, which can then be used to construct a
reward function for a model-based RL algorithm. The robot then learns the task
one stage at a time, automatically learning how to reset each stage to retry it
multiple times without human-provided resets. This makes the learning process
largely automatic, from intuitive task specification via a video to automated
training with minimal human intervention. We demonstrate that our approach is
capable of learning complex tasks, such as operating a coffee machine, directly
from raw image observations, requiring only 20 minutes to provide human
demonstrations and about 180 minutes of robot interaction.
