The promises and pitfalls of Stochastic Gradient Langevin Dynamics
  Stochastic Gradient Langevin Dynamics (SGLD) has emerged as a key MCMC
algorithm for Bayesian learning from large scale datasets. While SGLD with
decreasing step sizes converges weakly to the posterior distribution, the
algorithm is often used with a constant step size in practice and has
demonstrated successes in machine learning tasks. The current practice is to
set the step size inversely proportional to $N$ where $N$ is the number of
training samples. As $N$ becomes large, we show that the SGLD algorithm has an
invariant probability measure which significantly departs from the target
posterior and behaves like Stochastic Gradient Descent (SGD). This difference
is inherently due to the high variance of the stochastic gradients. Several
strategies have been suggested to reduce this effect; among them, SGLD Fixed
Point (SGLDFP) uses carefully designed control variates to reduce the variance
of the stochastic gradients. We show that SGLDFP gives approximate samples from
the posterior distribution, with an accuracy comparable to the Langevin Monte
Carlo (LMC) algorithm for a computational cost sublinear in the number of data
points. We provide a detailed analysis of the Wasserstein distances between
LMC, SGLD, SGLDFP and SGD and explicit expressions of the means and covariance
matrices of their invariant distributions. Our findings are supported by
limited numerical experiments.
