Function Space Particle Optimization for Bayesian Neural Networks
  While Bayesian neural networks (BNNs) have drawn increasing attention, their
posterior inference remains challenging, due to the high-dimensional and
over-parameterized nature. To address this issue, several highly flexible and
scalable variational inference procedures based on the idea of particle
optimization have been proposed. These methods directly optimize a set of
particles to approximate the target posterior. However, their application to
BNNs often yields sub-optimal performance, as such methods have a particular
failure mode on over-parameterized models. In this paper, we propose to solve
this issue by performing particle optimization directly in the space of
regression functions. We demonstrate through extensive experiments that our
method successfully overcomes this issue, and outperforms strong baselines in a
variety of tasks including prediction, defense against adversarial examples,
and reinforcement learning.
