De-randomizing MCMC dynamics with the diffusion Stein operator
  Approximate Bayesian inference estimates descriptors of an intractable target
distribution - in essence, an optimization problem within a family of
distributions. For example, Langevin dynamics (LD) extracts asymptotically
exact samples from a diffusion process because the time evolution of its
marginal distributions constitutes a curve that minimizes the KL-divergence via
steepest descent in the Wasserstein space. Parallel to LD, Stein variational
gradient descent (SVGD) similarly minimizes the KL, albeit endowed with a novel
Stein-Wasserstein distance, by deterministically transporting a set of particle
samples, thus de-randomizes the stochastic diffusion process. We propose
de-randomized kernel-based particle samplers to all diffusion-based samplers
known as MCMC dynamics. Following previous work in interpreting MCMC dynamics,
we equip the Stein-Wasserstein space with a fiber-Riemannian Poisson structure,
with the capacity of characterizing a fiber-gradient Hamiltonian flow that
simulates MCMC dynamics. Such dynamics discretizes into generalized SVGD
(GSVGD), a Stein-type deterministic particle sampler, with particle updates
coinciding with applying the diffusion Stein operator to a kernel function. We
demonstrate empirically that GSVGD can de-randomize complex MCMC dynamics,
which combine the advantages of auxiliary momentum variables and Riemannian
structure, while maintaining the high sample quality from an interacting
particle system.
