Imitation Learning from Imperfect Demonstration
  Imitation learning (IL) aims to learn an optimal policy from demonstrations.
However, such demonstrations are often imperfect since collecting optimal ones
is costly. To effectively learn from imperfect demonstrations, we propose a
novel approach that utilizes confidence scores, which describe the quality of
demonstrations. More specifically, we propose two confidence-based IL methods,
namely two-step importance weighting IL (2IWIL) and generative adversarial IL
with imperfect demonstration and confidence (IC-GAIL). We show that confidence
scores given only to a small portion of sub-optimal demonstrations
significantly improve the performance of IL both theoretically and empirically.
