Inference Compilation and Universal Probabilistic Programming
  We introduce a method for using deep neural networks to amortize the cost of
inference in models from the family induced by universal probabilistic
programming languages, establishing a framework that combines the strengths of
probabilistic programming and deep learning methods. We call what we do
"compilation of inference" because our method transforms a denotational
specification of an inference problem in the form of a probabilistic program
written in a universal programming language into a trained neural network
denoted in a neural network specification language. When at test time this
neural network is fed observational data and executed, it performs approximate
inference in the original model specified by the probabilistic program. Our
training objective and learning procedure are designed to allow the trained
neural network to be used as a proposal distribution in a sequential importance
sampling inference engine. We illustrate our method on mixture models and
Captcha solving and show significant speedups in the efficiency of inference.
