Efficient Gradient-Based Inference through Transformations between Bayes
  Nets and Neural Nets
  Hierarchical Bayesian networks and neural networks with stochastic hidden
units are commonly perceived as two separate types of models. We show that
either of these types of models can often be transformed into an instance of
the other, by switching between centered and differentiable non-centered
parameterizations of the latent variables. The choice of parameterization
greatly influences the efficiency of gradient-based posterior inference; we
show that they are often complementary to eachother, we clarify when each
parameterization is preferred and show how inference can be made robust. In the
non-centered form, a simple Monte Carlo estimator of the marginal likelihood
can be used for learning the parameters. Theoretical results are supported by
experiments.
