Pseudo-Spherical Contrastive Divergence
  Energy-based models (EBMs) offer flexible distribution parametrization.
However, due to the intractable partition function, they are typically trained
via contrastive divergence for maximum likelihood estimation. In this paper, we
propose pseudo-spherical contrastive divergence (PS-CD) to generalize maximum
likelihood learning of EBMs. PS-CD is derived from the maximization of a family
of strictly proper homogeneous scoring rules, which avoids the computation of
the intractable partition function and provides a generalized family of
learning objectives that include contrastive divergence as a special case.
Moreover, PS-CD allows us to flexibly choose various learning objectives to
train EBMs without additional computational cost or variational minimax
optimization. Theoretical analysis on the proposed method and extensive
experiments on both synthetic data and commonly used image datasets demonstrate
the effectiveness and modeling flexibility of PS-CD, as well as its robustness
to data contamination, thus showing its superiority over maximum likelihood and
$f$-EBMs.
