Stochastic Variational Inference with Gradient Linearization
  Variational inference has experienced a recent surge in popularity owing to
stochastic approaches, which have yielded practical tools for a wide range of
model classes. A key benefit is that stochastic variational inference obviates
the tedious process of deriving analytical expressions for closed-form variable
updates. Instead, one simply needs to derive the gradient of the log-posterior,
which is often much easier. Yet for certain model classes, the log-posterior
itself is difficult to optimize using standard gradient techniques. One such
example are random field models, where optimization based on gradient
linearization has proven popular, since it speeds up convergence significantly
and can avoid poor local optima. In this paper we propose stochastic
variational inference with gradient linearization (SVIGL). It is similarly
convenient as standard stochastic variational inference - all that is required
is a local linearization of the energy gradient. Its benefit over stochastic
variational inference with conventional gradient methods is a clear improvement
in convergence speed, while yielding comparable or even better variational
approximations in terms of KL divergence. We demonstrate the benefits of SVIGL
in three applications: Optical flow estimation, Poisson-Gaussian denoising, and
3D surface reconstruction.
