Grassmann Stein Variational Gradient Descent
  Stein variational gradient descent (SVGD) is a deterministic particle
inference algorithm that provides an efficient alternative to Markov chain
Monte Carlo. However, SVGD has been found to suffer from variance
underestimation when the dimensionality of the target distribution is high.
Recent developments have advocated projecting both the score function and the
data onto real lines to sidestep this issue, although this can severely
overestimate the epistemic (model) uncertainty. In this work, we propose
Grassmann Stein variational gradient descent (GSVGD) as an alternative
approach, which permits projections onto arbitrary dimensional subspaces.
Compared with other variants of SVGD that rely on dimensionality reduction,
GSVGD updates the projectors simultaneously for the score function and the
data, and the optimal projectors are determined through a coupled
Grassmann-valued diffusion process which explores favourable subspaces. Both
our theoretical and experimental results suggest that GSVGD enjoys efficient
state-space exploration in high-dimensional problems that have an intrinsic
low-dimensional structure.
