WeightScale: Interpreting Weight Change in Neural Networks
  Interpreting the learning dynamics of neural networks can provide useful
insights into how networks learn and the development of better training and
design approaches. We present an approach to interpret learning in neural
networks by measuring relative weight change on a per layer basis and
dynamically aggregating emerging trends through combination of dimensionality
reduction and clustering which allows us to scale to very deep networks. We use
this approach to investigate learning in the context of vision tasks across a
variety of state-of-the-art networks and provide insights into the learning
behavior of these networks, including how task complexity affects layer-wise
learning in deeper layers of networks.
