Universal Value Density Estimation for Imitation Learning and
  Goal-Conditioned Reinforcement Learning
  This work considers two distinct settings: imitation learning and
goal-conditioned reinforcement learning. In either case, effective solutions
require the agent to reliably reach a specified state (a goal), or set of
states (a demonstration). Drawing a connection between probabilistic long-term
dynamics and the desired value function, this work introduces an approach which
utilizes recent advances in density estimation to effectively learn to reach a
given state. As our first contribution, we use this approach for
goal-conditioned reinforcement learning and show that it is both efficient and
does not suffer from hindsight bias in stochastic domains. As our second
contribution, we extend the approach to imitation learning and show that it
achieves state-of-the art demonstration sample-efficiency on standard benchmark
tasks.
