Mind the Gap when Conditioning Amortised Inference in Sequential
  Latent-Variable Models
  Amortised inference enables scalable learning of sequential latent-variable
models (LVMs) with the evidence lower bound (ELBO). In this setting,
variational posteriors are often only partially conditioned. While the true
posteriors depend, e.g., on the entire sequence of observations, approximate
posteriors are only informed by past observations. This mimics the Bayesian
filter -- a mixture of smoothing posteriors. Yet, we show that the ELBO
objective forces partially-conditioned amortised posteriors to approximate
products of smoothing posteriors instead. Consequently, the learned generative
model is compromised. We demonstrate these theoretical findings in three
scenarios: traffic flow, handwritten digits, and aerial vehicle dynamics. Using
fully-conditioned approximate posteriors, performance improves in terms of
generative modelling and multi-step prediction.
