Inter-layer Information Similarity Assessment of Deep Neural Networks
  Via Topological Similarity and Persistence Analysis of Data Neighbour
  Dynamics
  The quantitative analysis of information structure through a deep neural
network (DNN) can unveil new insights into the theoretical performance of DNN
architectures. Two very promising avenues of research towards quantitative
information structure analysis are: 1) layer similarity (LS) strategies focused
on the inter-layer feature similarity, and 2) intrinsic dimensionality (ID)
strategies focused on layer-wise data dimensionality using pairwise
information. Inspired by both LS and ID strategies for quantitative information
structure analysis, we introduce two novel complimentary methods for
inter-layer information similarity assessment premised on the interesting idea
of studying a data sample's neighbourhood dynamics as it traverses through a
DNN. More specifically, we introduce the concept of Nearest Neighbour
Topological Similarity (NNTS) for quantifying the information topology
similarity between layers of a DNN. Furthermore, we introduce the concept of
Nearest Neighbour Topological Persistence (NNTP) for quantifying the
inter-layer persistence of data neighbourhood relationships throughout a DNN.
The proposed strategies facilitate the efficient inter-layer information
similarity assessment by leveraging only local topological information, and we
demonstrate their efficacy in this study by performing analysis on a deep
convolutional neural network architecture on image data to study the insights
that can be gained with respect to the theoretical performance of a DNN.
