VASE: Variational Assorted Surprise Exploration for Reinforcement
  Learning
  Exploration in environments with continuous control and sparse rewards
remains a key challenge in reinforcement learning (RL). Recently, surprise has
been used as an intrinsic reward that encourages systematic and efficient
exploration. We introduce a new definition of surprise and its RL
implementation named Variational Assorted Surprise Exploration (VASE). VASE
uses a Bayesian neural network as a model of the environment dynamics and is
trained using variational inference, alternately updating the accuracy of the
agent's model and policy. Our experiments show that in continuous control
sparse reward environments VASE outperforms other surprise-based exploration
techniques.
