Probability Density Estimation Based Imitation Learning
  Imitation Learning (IL) is an effective learning paradigm exploiting the
interactions between agents and environments. It does not require explicit
reward signals and instead tries to recover desired policies using expert
demonstrations. In general, IL methods can be categorized into Behavioral
Cloning (BC) and Inverse Reinforcement Learning (IRL). In this work, a novel
reward function based on probability density estimation is proposed for IRL,
which can significantly reduce the complexity of existing IRL methods.
Furthermore, we prove that the theoretically optimal policy derived from our
reward function is identical to the expert policy as long as it is
deterministic. Consequently, an IRL problem can be gracefully transformed into
a probability density estimation problem. Based on the proposed reward
function, we present a "watch-try-learn" style framework named Probability
Density Estimation based Imitation Learning (PDEIL), which can work in both
discrete and continuous action spaces. Finally, comprehensive experiments in
the Gym environment show that PDEIL is much more efficient than existing
algorithms in recovering rewards close to the ground truth.
