Variational inference for Monte Carlo objectives
  Recent progress in deep latent variable models has largely been driven by the
development of flexible and scalable variational inference methods. Variational
training of this type involves maximizing a lower bound on the log-likelihood,
using samples from the variational posterior to compute the required gradients.
Recently, Burda et al. (2016) have derived a tighter lower bound using a
multi-sample importance sampling estimate of the likelihood and showed that
optimizing it yields models that use more of their capacity and achieve higher
likelihoods. This development showed the importance of such multi-sample
objectives and explained the success of several related approaches.
  We extend the multi-sample approach to discrete latent variables and analyze
the difficulty encountered when estimating the gradients involved. We then
develop the first unbiased gradient estimator designed for importance-sampled
objectives and evaluate it at training generative and structured output
prediction models. The resulting estimator, which is based on low-variance
per-sample learning signals, is both simpler and more effective than the NVIL
estimator proposed for the single-sample variational objective, and is
competitive with the currently used biased estimators.
