Contrastive Divergence Learning is a Time Reversal Adversarial Game
  Contrastive divergence (CD) learning is a classical method for fitting
unnormalized statistical models to data samples. Despite its wide-spread use,
the convergence properties of this algorithm are still not well understood. The
main source of difficulty is an unjustified approximation which has been used
to derive the gradient of the loss. In this paper, we present an alternative
derivation of CD that does not require any approximation and sheds new light on
the objective that is actually being optimized by the algorithm. Specifically,
we show that CD is an adversarial learning procedure, where a discriminator
attempts to classify whether a Markov chain generated from the model has been
time-reversed. Thus, although predating generative adversarial networks (GANs)
by more than a decade, CD is, in fact, closely related to these techniques. Our
derivation settles well with previous observations, which have concluded that
CD's update steps cannot be expressed as the gradients of any fixed objective
function. In addition, as a byproduct, our derivation reveals a simple
correction that can be used as an alternative to Metropolis-Hastings rejection,
which is required when the underlying Markov chain is inexact (e.g. when using
Langevin dynamics with a large step).
