Scalable Bayesian Inverse Reinforcement Learning
  Bayesian inference over the reward presents an ideal solution to the
ill-posed nature of the inverse reinforcement learning problem. Unfortunately
current methods generally do not scale well beyond the small tabular setting
due to the need for an inner-loop MDP solver, and even non-Bayesian methods
that do themselves scale often require extensive interaction with the
environment to perform well, being inappropriate for high stakes or costly
applications such as healthcare. In this paper we introduce our method,
Approximate Variational Reward Imitation Learning (AVRIL), that addresses both
of these issues by jointly learning an approximate posterior distribution over
the reward that scales to arbitrarily complicated state spaces alongside an
appropriate policy in a completely offline manner through a variational
approach to said latent reward. Applying our method to real medical data
alongside classic control simulations, we demonstrate Bayesian reward inference
in environments beyond the scope of current methods, as well as task
performance competitive with focused offline imitation learning algorithms.
