Seeing Differently, Acting Similarly: Heterogeneously Observable
  Imitation Learning
  In many real-world imitation learning tasks, the demonstrator and the learner
have to act under totally different observation spaces. This situation brings
significant obstacles to existing imitation learning approaches, since most of
them learn policies under homogeneous observation spaces. On the other hand,
previous studies under different observation spaces have strong assumptions
that these two observation spaces coexist during the entire learning process.
However, in reality, the observation coexistence will be limited due to the
high cost of acquiring expert observations. In this work, we study this
challenging problem with limited observation coexistence under heterogeneous
observations: Heterogeneously Observable Imitation Learning (HOIL). We identify
two underlying issues in HOIL, i.e., the dynamics mismatch and the support
mismatch, and further propose the Importance Weighting with REjection (IWRE)
algorithm based on importance-weighting and learning with rejection to solve
HOIL problems. Experimental results show that IWRE can successfully solve
various HOIL tasks, including the challenging tasks of transforming the
vision-based demonstrations to random access memory (RAM)-based policies in the
Atari domain, even with limited visual observations.
