Domain Adaptive Imitation Learning
  We study the question of how to imitate tasks across domains with
discrepancies such as embodiment, viewpoint, and dynamics mismatch. Many prior
works require paired, aligned demonstrations and an additional RL step that
requires environment interactions. However, paired, aligned demonstrations are
seldom obtainable and RL procedures are expensive. We formalize the Domain
Adaptive Imitation Learning (DAIL) problem, which is a unified framework for
imitation learning in the presence of viewpoint, embodiment, and dynamics
mismatch. Informally, DAIL is the process of learning how to perform a task
optimally, given demonstrations of the task in a distinct domain. We propose a
two step approach to DAIL: alignment followed by adaptation. In the alignment
step we execute a novel unsupervised MDP alignment algorithm, Generative
Adversarial MDP Alignment (GAMA), to learn state and action correspondences
from \emph{unpaired, unaligned} demonstrations. In the adaptation step we
leverage the correspondences to zero-shot imitate tasks across domains. To
describe when DAIL is feasible via alignment and adaptation, we introduce a
theory of MDP alignability. We experimentally evaluate GAMA against baselines
in embodiment, viewpoint, and dynamics mismatch scenarios where aligned
demonstrations don't exist and show the effectiveness of our approach.
