On importance-weighted autoencoders
  The importance weighted autoencoder (IWAE) (Burda et al., 2016) is a popular
variational-inference method which achieves a tighter evidence bound (and hence
a lower bias) than standard variational autoencoders by optimising a
multi-sample objective, i.e. an objective that is expressible as an integral
over $K > 1$ Monte Carlo samples. Unfortunately, IWAE crucially relies on the
availability of reparametrisations and even if these exist, the multi-sample
objective leads to inference-network gradients which break down as $K$ is
increased (Rainforth et al., 2018). This breakdown can only be circumvented by
removing high-variance score-function terms, either by heuristically ignoring
them (which yields the 'sticking-the-landing' IWAE (IWAE-STL) gradient from
Roeder et al. (2017)) or through an identity from Tucker et al. (2019) (which
yields the 'doubly-reparametrised' IWAE (IWAE-DREG) gradient). In this work, we
argue that directly optimising the proposal distribution in importance sampling
as in the reweighted wake-sleep (RWS) algorithm from Bornschein & Bengio (2015)
is preferable to optimising IWAE-type multi-sample objectives. To formalise
this argument, we introduce an adaptive-importance sampling framework termed
adaptive importance sampling for learning (AISLE) which slightly generalises
the RWS algorithm. We then show that AISLE admits IWAE-STL and IWAE-DREG (i.e.
the IWAE-gradients which avoid breakdown) as special cases.
