Imitation Learning from Observations by Minimizing Inverse Dynamics
  Disagreement
  This paper studies Learning from Observations (LfO) for imitation learning
with access to state-only demonstrations. In contrast to Learning from
Demonstration (LfD) that involves both action and state supervision, LfO is
more practical in leveraging previously inapplicable resources (e.g. videos),
yet more challenging due to the incomplete expert guidance. In this paper, we
investigate LfO and its difference with LfD in both theoretical and practical
perspectives. We first prove that the gap between LfD and LfO actually lies in
the disagreement of inverse dynamics models between the imitator and the
expert, if following the modeling approach of GAIL. More importantly, the upper
bound of this gap is revealed by a negative causal entropy which can be
minimized in a model-free way. We term our method as
Inverse-Dynamics-Disagreement-Minimization (IDDM) which enhances the
conventional LfO method through further bridging the gap to LfD. Considerable
empirical results on challenging benchmarks indicate that our method attains
consistent improvements over other LfO counterparts.
