No Need for Interactions: Robust Model-Based Imitation Learning using
  Neural ODE
  Interactions with either environments or expert policies during training are
needed for most of the current imitation learning (IL) algorithms. For IL
problems with no interactions, a typical approach is Behavior Cloning (BC).
However, BC-like methods tend to be affected by distribution shift. To mitigate
this problem, we come up with a Robust Model-Based Imitation Learning (RMBIL)
framework that casts imitation learning as an end-to-end differentiable
nonlinear closed-loop tracking problem. RMBIL applies Neural ODE to learn a
precise multi-step dynamics and a robust tracking controller via Nonlinear
Dynamics Inversion (NDI) algorithm. Then, the learned NDI controller will be
combined with a trajectory generator, a conditional VAE, to imitate an expert's
behavior. Theoretical derivation shows that the controller network can
approximate an NDI when minimizing the training loss of Neural ODE. Experiments
on Mujoco tasks also demonstrate that RMBIL is competitive to the
state-of-the-art generative adversarial method (GAIL) and achieves at least 30%
performance gain over BC in uneven surfaces.
