Error Bounds of Imitating Policies and Environments
  Imitation learning trains a policy by mimicking expert demonstrations.
Various imitation methods were proposed and empirically evaluated, meanwhile,
their theoretical understanding needs further studies. In this paper, we
firstly analyze the value gap between the expert policy and imitated policies
by two imitation methods, behavioral cloning and generative adversarial
imitation. The results support that generative adversarial imitation can reduce
the compounding errors compared to behavioral cloning, and thus has a better
sample complexity. Noticed that by considering the environment transition model
as a dual agent, imitation learning can also be used to learn the environment
model. Therefore, based on the bounds of imitating policies, we further analyze
the performance of imitating environments. The results show that environment
models can be more effectively imitated by generative adversarial imitation
than behavioral cloning, suggesting a novel application of adversarial
imitation for model-based reinforcement learning. We hope these results could
inspire future advances in imitation learning and model-based reinforcement
learning.
