Dynamical mean-field theory for stochastic gradient descent in Gaussian
  mixture classification
  We analyze in a closed form the learning dynamics of stochastic gradient
descent (SGD) for a single-layer neural network classifying a high-dimensional
Gaussian mixture where each cluster is assigned one of two labels. This problem
provides a prototype of a non-convex loss landscape with interpolating regimes
and a large generalization gap. We define a particular stochastic process for
which SGD can be extended to a continuous-time limit that we call stochastic
gradient flow. In the full-batch limit, we recover the standard gradient flow.
We apply dynamical mean-field theory from statistical physics to track the
dynamics of the algorithm in the high-dimensional limit via a self-consistent
stochastic process. We explore the performance of the algorithm as a function
of the control parameters shedding light on how it navigates the loss
landscape.
