No MCMC for me: Amortized sampling for fast and stable training of
  energy-based models
  Energy-Based Models (EBMs) present a flexible and appealing way to represent
uncertainty. Despite recent advances, training EBMs on high-dimensional data
remains a challenging problem as the state-of-the-art approaches are costly,
unstable, and require considerable tuning and domain expertise to apply
successfully. In this work, we present a simple method for training EBMs at
scale which uses an entropy-regularized generator to amortize the MCMC sampling
typically used in EBM training. We improve upon prior MCMC-based entropy
regularization methods with a fast variational approximation. We demonstrate
the effectiveness of our approach by using it to train tractable likelihood
models. Next, we apply our estimator to the recently proposed Joint Energy
Model (JEM), where we match the original performance with faster and stable
training. This allows us to extend JEM models to semi-supervised classification
on tabular data from a variety of continuous domains.
