Adversarial Skill Networks: Unsupervised Robot Skill Learning from Video
  Key challenges for the deployment of reinforcement learning (RL) agents in
the real world are the discovery, representation and reuse of skills in the
absence of a reward function. To this end, we propose a novel approach to learn
a task-agnostic skill embedding space from unlabeled multi-view videos. Our
method learns a general skill embedding independently from the task context by
using an adversarial loss. We combine a metric learning loss, which utilizes
temporal video coherence to learn a state representation, with an entropy
regularized adversarial skill-transfer loss. The metric learning loss learns a
disentangled representation by attracting simultaneous viewpoints of the same
observations and repelling visually similar frames from temporal neighbors. The
adversarial skill-transfer loss enhances re-usability of learned skill
embeddings over multiple task domains. We show that the learned embedding
enables training of continuous control policies to solve novel tasks that
require the interpolation of previously seen skills. Our extensive evaluation
with both simulation and real world data demonstrates the effectiveness of our
method in learning transferable skills from unlabeled interaction videos and
composing them for new tasks. Code, pretrained models and dataset are available
at http://robotskills.cs.uni-freiburg.de
