Elements of Sequential Monte Carlo
  A core problem in statistics and probabilistic machine learning is to compute
probability distributions and expectations. This is the fundamental problem of
Bayesian statistics and machine learning, which frames all inference as
expectations with respect to the posterior distribution. The key challenge is
to approximate these intractable expectations. In this tutorial, we review
sequential Monte Carlo (SMC), a random-sampling-based class of methods for
approximate inference. First, we explain the basics of SMC, discuss practical
issues, and review theoretical results. We then examine two of the main user
design choices: the proposal distributions and the so called intermediate
target distributions. We review recent results on how variational inference and
amortization can be used to learn efficient proposals and target distributions.
Next, we discuss the SMC estimate of the normalizing constant, how this can be
used for pseudo-marginal inference and inference evaluation. Throughout the
tutorial we illustrate the use of SMC on various models commonly used in
machine learning, such as stochastic recurrent neural networks, probabilistic
graphical models, and probabilistic programs.
