Subspace Clustering Based Analysis of Neural Networks
  Tools to analyze the latent space of deep neural networks provide a step
towards better understanding them. In this work, we motivate sparse subspace
clustering (SSC) with an aim to learn affinity graphs from the latent structure
of a given neural network layer trained over a set of inputs. We then use tools
from Community Detection to quantify structures present in the input. These
experiments reveal that as we go deeper in a network, inputs tend to have an
increasing affinity to other inputs of the same class. Subsequently, we utilise
matrix similarity measures to perform layer-wise comparisons between affinity
graphs. In doing so we first demonstrate that when comparing a given layer
currently under training to its final state, the shallower the layer of the
network, the quicker it is to converge than the deeper layers. When performing
a pairwise analysis of the entire network architecture, we observe that, as the
network increases in size, it reorganises from a state where each layer is
moderately similar to its neighbours, to a state where layers within a block
have high similarity than to layers in other blocks. Finally, we analyze the
learned affinity graphs of the final convolutional layer of the network and
demonstrate how an input's local neighbourhood affects its classification by
the network.
