Sparse generative modeling via parameter-reduction of Boltzmann
  machines: application to protein-sequence families
  Boltzmann machines (BM) are widely used as generative models. For example,
pairwise Potts models (PM), which are instances of the BM class, provide
accurate statistical models of families of evolutionarily related protein
sequences. Their parameters are the local fields, which describe site-specific
patterns of amino-acid conservation, and the two-site couplings, which mirror
the coevolution between pairs of sites. This coevolution reflects structural
and functional constraints acting on protein sequences during evolution. The
most conservative choice to describe the coevolution signal is to include all
possible two-site couplings into the PM. This choice, typical of what is known
as Direct Coupling Analysis, has been successful for predicting residue
contacts in the three-dimensional structure, mutational effects, and in
generating new functional sequences. However, the resulting PM suffers from
important over-fitting effects: many couplings are small, noisy and hardly
interpretable; the PM is close to a critical point, meaning that it is highly
sensitive to small parameter perturbations. In this work, we introduce a
general parameter-reduction procedure for BMs, via a controlled iterative
decimation of the less statistically significant couplings, identified by an
information-based criterion that selects either weak or statistically
unsupported couplings. For several protein families, our procedure allows one
to remove more than $90\%$ of the PM couplings, while preserving the predictive
and generative properties of the original dense PM, and the resulting model is
far away from criticality, hence more robust to noise.
