Robust Generative Adversarial Imitation Learning via Local Lipschitzness
  We explore methodologies to improve the robustness of generative adversarial
imitation learning (GAIL) algorithms to observation noise. Towards this
objective, we study the effect of local Lipschitzness of the discriminator and
the generator on the robustness of policies learned by GAIL. In many robotics
applications, the learned policies by GAIL typically suffer from a degraded
performance at test time since the observations from the environment might be
corrupted by noise. Hence, robustifying the learned policies against the
observation noise is of critical importance. To this end, we propose a
regularization method to induce local Lipschitzness in the generator and the
discriminator of adversarial imitation learning methods. We show that the
modified objective leads to learning significantly more robust policies.
Moreover, we demonstrate --- both theoretically and experimentally --- that
training a locally Lipschitz discriminator leads to a locally Lipschitz
generator, thereby improving the robustness of the resultant policy. We perform
extensive experiments on simulated robot locomotion environments from the
MuJoCo suite that demonstrate the proposed method learns policies that
significantly outperform the state-of-the-art generative adversarial imitation
learning algorithm when applied to test scenarios with noise-corrupted
observations.
