The edge of chaos: quantum field theory and deep neural networks
  We explicitly construct the quantum field theory corresponding to a general
class of deep neural networks encompassing both recurrent and feedforward
architectures. We first consider the mean-field theory (MFT) obtained as the
leading saddlepoint in the action, and derive the condition for criticality via
the largest Lyapunov exponent. We then compute the loop corrections to the
correlation function in a perturbative expansion in the ratio of depth $T$ to
width $N$, and find a precise analogy with the well-studied $O(N)$ vector
model, in which the variance of the weight initializations plays the role of
the 't Hooft coupling. In particular, we compute both the $\mathcal{O}(1)$
corrections quantifying fluctuations from typicality in the ensemble of
networks, and the subleading $\mathcal{O}(T/N)$ corrections due to finite-width
effects. These provide corrections to the correlation length that controls the
depth to which information can propagate through the network, and thereby sets
the scale at which such networks are trainable by gradient descent. Our
analysis provides a first-principles approach to the rapidly emerging NN-QFT
correspondence, and opens several interesting avenues to the study of
criticality in deep neural networks.
