Gaussian-Spherical Restricted Boltzmann Machines
  We consider a special type of Restricted Boltzmann machine (RBM), namely a
Gaussian-spherical RBM where the visible units have Gaussian priors while the
vector of hidden variables is constrained to stay on an ${\mathbbm L}_2$
sphere. The spherical constraint having the advantage to admit exact asymptotic
treatments, various scaling regimes are explicitly identified based solely on
the spectral properties of the coupling matrix (also called weight matrix of
the RBM). Incidentally these happen to be formally related to similar scaling
behaviours obtained in a different context dealing with spatial condensation of
zero range processes. More specifically, when the spectrum of the coupling
matrix is doubly degenerated an exact treatment can be proposed to deal with
finite size effects. Interestingly the known parallel between the ferromagnetic
transition of the spherical model and the Bose-Einstein condensation can be
made explicit in that case. More importantly this gives us the ability to
extract all needed response functions with arbitrary precision for the training
algorithm of the RBM. This allows us then to numerically integrate the dynamics
of the spectrum of the weight matrix during learning in a precise way. This
dynamics reveals in particular a sequential emergence of modes from the
Marchenko-Pastur bulk of singular vectors of the coupling matrix.
