Variational Inference with Locally Enhanced Bounds for Hierarchical
  Models
  Hierarchical models represent a challenging setting for inference algorithms.
MCMC methods struggle to scale to large models with many local variables and
observations, and variational inference (VI) may fail to provide accurate
approximations due to the use of simple variational families. Some variational
methods (e.g. importance weighted VI) integrate Monte Carlo methods to give
better accuracy, but these tend to be unsuitable for hierarchical models, as
they do not allow for subsampling and their performance tends to degrade for
high dimensional models. We propose a new family of variational bounds for
hierarchical models, based on the application of tightening methods (e.g.
importance weighting) separately for each group of local random variables. We
show that our approach naturally allows the use of subsampling to get unbiased
gradients, and that it fully leverages the power of methods that build tighter
lower bounds by applying them independently in lower dimensional spaces,
leading to better results and more accurate posterior approximations than
relevant baselines.
