XIRL: Cross-embodiment Inverse Reinforcement Learning
  We investigate the visual cross-embodiment imitation setting, in which agents
learn policies from videos of other agents (such as humans) demonstrating the
same task, but with stark differences in their embodiments -- shape, actions,
end-effector dynamics, etc. In this work, we demonstrate that it is possible to
automatically discover and learn vision-based reward functions from
cross-embodiment demonstration videos that are robust to these differences.
Specifically, we present a self-supervised method for Cross-embodiment Inverse
Reinforcement Learning (XIRL) that leverages temporal cycle-consistency
constraints to learn deep visual embeddings that capture task progression from
offline videos of demonstrations across multiple expert agents, each performing
the same task differently due to embodiment differences. Prior to our work,
producing rewards from self-supervised embeddings typically required alignment
with a reference trajectory, which may be difficult to acquire under stark
embodiment differences. We show empirically that if the embeddings are aware of
task progress, simply taking the negative distance between the current state
and goal state in the learned embedding space is useful as a reward for
training policies with reinforcement learning. We find our learned reward
function not only works for embodiments seen during training, but also
generalizes to entirely new embodiments. Additionally, when transferring
real-world human demonstrations to a simulated robot, we find that XIRL is more
sample efficient than current best methods. Qualitative results, code, and
datasets are available at https://x-irl.github.io
