Automatic Reparameterisation of Probabilistic Programs
  Probabilistic programming has emerged as a powerful paradigm in statistics,
applied science, and machine learning: by decoupling modelling from inference,
it promises to allow modellers to directly reason about the processes
generating data. However, the performance of inference algorithms can be
dramatically affected by the parameterisation used to express a model,
requiring users to transform their programs in non-intuitive ways. We argue for
automating these transformations, and demonstrate that mechanisms available in
recent modeling frameworks can implement non-centring and related
reparameterisations. This enables new inference algorithms, and we propose two:
a simple approach using interleaved sampling and a novel variational
formulation that searches over a continuous space of parameterisations. We show
that these approaches enable robust inference across a range of models, and can
yield more efficient samplers than the best fixed parameterisation.
