Isotropic SGD: a Practical Approach to Bayesian Posterior Sampling
  In this work we define a unified mathematical framework to deepen our
understanding of the role of stochastic gradient (SG) noise on the behavior of
Markov chain Monte Carlo sampling (SGMCMC) algorithms.
  Our formulation unlocks the design of a novel, practical approach to
posterior sampling, which makes the SG noise isotropic using a fixed learning
rate that we determine analytically, and that requires weaker assumptions than
existing algorithms. In contrast, the common traits of existing \sgmcmc
algorithms is to approximate the isotropy condition either by drowning the
gradients in additive noise (annealing the learning rate) or by making
restrictive assumptions on the \sg noise covariance and the geometry of the
loss landscape.
  Extensive experimental validations indicate that our proposal is competitive
with the state-of-the-art on \sgmcmc, while being much more practical to use.
