Variational Inference MPC for Bayesian Model-based Reinforcement
  Learning
  In recent studies on model-based reinforcement learning (MBRL), incorporating
uncertainty in forward dynamics is a state-of-the-art strategy to enhance
learning performance, making MBRLs competitive to cutting-edge model free
methods, especially in simulated robotics tasks. Probabilistic ensembles with
trajectory sampling (PETS) is a leading type of MBRL, which employs Bayesian
inference to dynamics modeling and model predictive control (MPC) with
stochastic optimization via the cross entropy method (CEM). In this paper, we
propose a novel extension to the uncertainty-aware MBRL. Our main contributions
are twofold: Firstly, we introduce a variational inference MPC, which
reformulates various stochastic methods, including CEM, in a Bayesian fashion.
Secondly, we propose a novel instance of the framework, called probabilistic
action ensembles with trajectory sampling (PaETS). As a result, our Bayesian
MBRL can involve multimodal uncertainties both in dynamics and optimal
trajectories. In comparison to PETS, our method consistently improves
asymptotic performance on several challenging locomotion tasks.
