A Non-Asymptotic Analysis for Stein Variational Gradient Descent
  We study the Stein Variational Gradient Descent (SVGD) algorithm, which
optimises a set of particles to approximate a target probability distribution
$\pi\propto e^{-V}$ on $\mathbb{R}^d$. In the population limit, SVGD performs
gradient descent in the space of probability distributions on the KL divergence
with respect to $\pi$, where the gradient is smoothed through a kernel integral
operator. In this paper, we provide a novel finite time analysis for the SVGD
algorithm. We provide a descent lemma establishing that the algorithm decreases
the objective at each iteration, and rates of convergence for the average Stein
Fisher divergence (also referred to as Kernel Stein Discrepancy). We also
provide a convergence result of the finite particle system corresponding to the
practical implementation of SVGD to its population version.
