Accelerating Stochastic Probabilistic Inference
  Recently, Stochastic Variational Inference (SVI) has been increasingly
attractive thanks to its ability to find good posterior approximations of
probabilistic models. It optimizes the variational objective with stochastic
optimization, following noisy estimates of the natural gradient. However,
almost all the state-of-the-art SVI algorithms are based on first-order
optimization algorithm and often suffer from poor convergence rate. In this
paper, we bridge the gap between second-order methods and stochastic
variational inference by proposing a second-order based stochastic variational
inference approach. In particular, firstly we derive the Hessian matrix of the
variational objective. Then we devise two numerical schemes to implement
second-order SVI efficiently. Thorough empirical evaluations are investigated
on both synthetic and real dataset to backup both the effectiveness and
efficiency of the proposed approach.
