Efficient Approximate Inference with Walsh-Hadamard Variational
  Inference
  Variational inference offers scalable and flexible tools to tackle
intractable Bayesian inference of modern statistical models like Bayesian
neural networks and Gaussian processes. For largely over-parameterized models,
however, the over-regularization property of the variational objective makes
the application of variational inference challenging. Inspired by the
literature on kernel methods, and in particular on structured approximations of
distributions of random matrices, this paper proposes Walsh-Hadamard
Variational Inference, which uses Walsh-Hadamard-based factorization strategies
to reduce model parameterization, accelerate computations, and increase the
expressiveness of the approximate posterior beyond fully factorized ones.
