Approximating exponential family models (not single distributions) with
  a two-network architecture
  Recently much attention has been paid to deep generative models, since they
have been used to great success for variational inference, generation of
complex data types, and more. In most all of these settings, the goal has been
to find a particular member of that model family: optimized parameters index a
distribution that is close (via a divergence or classification metric) to a
target distribution. Much less attention, however, has been paid to the problem
of learning a model itself. Here we introduce a two-network architecture and
optimization procedure for learning intractable exponential family models (not
a single distribution from those models). These exponential families are
learned accurately, allowing operations like posterior inference to be executed
directly and generically with an input choice of natural parameters, rather
than performing inference via optimization for each particular distribution
within that model.
