Good Initializations of Variational Bayes for Deep Models
  Stochastic variational inference is an established way to carry out
approximate Bayesian inference for deep models. While there have been effective
proposals for good initializations for loss minimization in deep learning, far
less attention has been devoted to the issue of initialization of stochastic
variational inference. We address this by proposing a novel layer-wise
initialization strategy based on Bayesian linear models. The proposed method is
extensively validated on regression and classification tasks, including
Bayesian DeepNets and ConvNets, showing faster and better convergence compared
to alternatives inspired by the literature on initializations for loss
minimization.
