Efficient Semi-Implicit Variational Inference
  In this paper, we propose CI-VI an efficient and scalable solver for
semi-implicit variational inference (SIVI). Our method, first, maps SIVI's
evidence lower bound (ELBO) to a form involving a nonlinear functional nesting
of expected values and then develops a rigorous optimiser capable of correctly
handling bias inherent to nonlinear nested expectations using an
extrapolation-smoothing mechanism coupled with gradient sketching. Our
theoretical results demonstrate convergence to a stationary point of the ELBO
in general non-convex settings typically arising when using deep network models
and an order of $O(t^{-\frac{4}{5}})$ gradient-bias-vanishing rate. We believe
these results generalise beyond the specific nesting arising from SIVI to other
forms. Finally, in a set of experiments, we demonstrate the effectiveness of
our algorithm in approximating complex posteriors on various data-sets
including those from natural language processing.
