CWAE-IRL: Formulating a supervised approach to Inverse Reinforcement
  Learning problem
  Inverse reinforcement learning (IRL) is used to infer the reward function
from the actions of an expert running a Markov Decision Process (MDP). A novel
approach using variational inference for learning the reward function is
proposed in this research. Using this technique, the intractable posterior
distribution of the continuous latent variable (the reward function in this
case) is analytically approximated to appear to be as close to the prior belief
while trying to reconstruct the future state conditioned on the current state
and action. The reward function is derived using a well-known deep generative
model known as Conditional Variational Auto-encoder (CVAE) with Wasserstein
loss function, thus referred to as Conditional Wasserstein Auto-encoder-IRL
(CWAE-IRL), which can be analyzed as a combination of the backward and forward
inference. This can then form an efficient alternative to the previous
approaches to IRL while having no knowledge of the system dynamics of the
agent. Experimental results on standard benchmarks such as objectworld and
pendulum show that the proposed algorithm can effectively learn the latent
reward function in complex, high-dimensional environments.
