Lagrangian Manifold Monte Carlo on Monge Patches
  The efficiency of Markov Chain Monte Carlo (MCMC) depends on how the
underlying geometry of the problem is taken into account. For distributions
with strongly varying curvature, Riemannian metrics help in efficient
exploration of the target distribution. Unfortunately, they have significant
computational overhead due to e.g. repeated inversion of the metric tensor, and
current geometric MCMC methods using the Fisher information matrix to induce
the manifold are in practice slow. We propose a new alternative Riemannian
metric for MCMC, by embedding the target distribution into a higher-dimensional
Euclidean space as a Monge patch and using the induced metric determined by
direct geometric reasoning. Our metric only requires first-order gradient
information and has fast inverse and determinants, and allows reducing the
computational complexity of individual iterations from cubic to quadratic in
the problem dimensionality. We demonstrate how Lagrangian Monte Carlo in this
metric efficiently explores the target distributions.
