Learning of Discrete Graphical Models with Neural Networks
  Graphical models are widely used in science to represent joint probability
distributions with an underlying conditional dependence structure. The inverse
problem of learning a discrete graphical model given i.i.d samples from its
joint distribution can be solved with near-optimal sample complexity using a
convex optimization method known as Generalized Regularized Interaction
Screening Estimator (GRISE). But the computational cost of GRISE becomes
prohibitive when the energy function of the true graphical model has
higher-order terms. We introduce NeurISE, a neural net based algorithm for
graphical model learning, to tackle this limitation of GRISE. We use neural
nets as function approximators in an Interaction Screening objective function.
The optimization of this objective then produces a neural-net representation
for the conditionals of the graphical model. NeurISE algorithm is seen to be a
better alternative to GRISE when the energy function of the true model has a
high order with a high degree of symmetry. In these cases NeurISE is able to
find the correct parsimonious representation for the conditionals without being
fed any prior information about the true model. NeurISE can also be used to
learn the underlying structure of the true model with some simple modifications
to its training procedure. In addition, we also show a variant of NeurISE that
can be used to learn a neural net representation for the full energy function
of the true model.
