Characterizing Learning Dynamics of Deep Neural Networks via Complex
  Networks
  In this paper, we interpret Deep Neural Networks with Complex Network Theory.
Complex Network Theory (CNT) represents Deep Neural Networks (DNNs) as directed
weighted graphs to study them as dynamical systems. We efficiently adapt CNT
measures to examine the evolution of the learning process of DNNs with
different initializations and architectures: we introduce metrics for
nodes/neurons and layers, namely Nodes Strength and Layers Fluctuation. Our
framework distills trends in the learning dynamics and separates low from high
accurate networks. We characterize populations of neural networks (ensemble
analysis) and single instances (individual analysis). We tackle standard
problems of image recognition, for which we show that specific learning
dynamics are indistinguishable when analysed through the solely Link-Weights
analysis. Further, Nodes Strength and Layers Fluctuations make unprecedented
behaviours emerge: accurate networks, when compared to under-trained models,
show substantially divergent distributions with the greater extremity of
deviations. On top of this study, we provide an efficient implementation of the
CNT metrics for both Convolutional and Fully Connected Networks, to fasten the
research in this direction.
