Filtering Variational Objectives
  When used as a surrogate objective for maximum likelihood estimation in
latent variable models, the evidence lower bound (ELBO) produces
state-of-the-art results. Inspired by this, we consider the extension of the
ELBO to a family of lower bounds defined by a particle filter's estimator of
the marginal likelihood, the filtering variational objectives (FIVOs). FIVOs
take the same arguments as the ELBO, but can exploit a model's sequential
structure to form tighter bounds. We present results that relate the tightness
of FIVO's bound to the variance of the particle filter's estimator by
considering the generic case of bounds defined as log-transformed likelihood
estimators. Experimentally, we show that training with FIVO results in
substantial improvements over training the same model architecture with the
ELBO on sequential data.
