Hamiltonian Dynamics with Non-Newtonian Momentum for Rapid Sampling
  Sampling from an unnormalized probability distribution is a fundamental
problem in machine learning with applications including Bayesian modeling,
latent factor inference, and energy-based model training. After decades of
research, variations of MCMC remain the default approach to sampling despite
slow convergence. Auxiliary neural models can learn to speed up MCMC, but the
overhead for training the extra model can be prohibitive. We propose a
fundamentally different approach to this problem via a new Hamiltonian dynamics
with a non-Newtonian momentum. In contrast to MCMC approaches like Hamiltonian
Monte Carlo, no stochastic step is required. Instead, the proposed
deterministic dynamics in an extended state space exactly sample the target
distribution, specified by an energy function, under an assumption of
ergodicity. Alternatively, the dynamics can be interpreted as a normalizing
flow that samples a specified energy model without training. The proposed
Energy Sampling Hamiltonian (ESH) dynamics have a simple form that can be
solved with existing ODE solvers, but we derive a specialized solver that
exhibits much better performance. ESH dynamics converge faster than their MCMC
competitors enabling faster, more stable training of neural network energy
models.
