Learning Equivariant Energy Based Models with Equivariant Stein
  Variational Gradient Descent
  We focus on the problem of efficient sampling and learning of probability
densities by incorporating symmetries in probabilistic models. We first
introduce Equivariant Stein Variational Gradient Descent algorithm -- an
equivariant sampling method based on Stein's identity for sampling from
densities with symmetries. Equivariant SVGD explicitly incorporates symmetry
information in a density through equivariant kernels which makes the resultant
sampler efficient both in terms of sample complexity and the quality of
generated samples. Subsequently, we define equivariant energy based models to
model invariant densities that are learned using contrastive divergence. By
utilizing our equivariant SVGD for training equivariant EBMs, we propose new
ways of improving and scaling up training of energy based models. We apply
these equivariant energy models for modelling joint densities in regression and
classification tasks for image datasets, many-body particle systems and
molecular structure generation.
