Scale-invariant Feature Extraction of Neural Network and Renormalization
  Group Flow
  Theoretical understanding of how deep neural network (DNN) extracts features
from input images is still unclear, but it is widely believed that the
extraction is performed hierarchically through a process of coarse-graining. It
reminds us of the basic concept of renormalization group (RG) in statistical
physics. In order to explore possible relations between DNN and RG, we use the
Restricted Boltzmann machine (RBM) applied to Ising model and construct a flow
of model parameters (in particular, temperature) generated by the RBM. We show
that the unsupervised RBM trained by spin configurations at various
temperatures from $T=0$ to $T=6$ generates a flow along which the temperature
approaches the critical value $T_c=2.27$. This behavior is opposite to the
typical RG flow of the Ising model. By analyzing various properties of the
weight matrices of the trained RBM, we discuss why it flows towards $T_c$ and
how the RBM learns to extract features of spin configurations.
