Quantitative Performance Assessment of CNN Units via Topological Entropy
  Calculation
  Identifying the status of individual network units is critical for
understanding the mechanism of convolutional neural networks (CNNs). However,
it is still challenging to reliably give a general indication of unit status,
especially for units in different network models. To this end, we propose a
novel method for quantitatively clarifying the status of single unit in CNN
using algebraic topological tools. Unit status is indicated via the calculation
of a defined topological-based entropy, called feature entropy, which measures
the degree of chaos of the global spatial pattern hidden in the unit for a
category. In this way, feature entropy could provide an accurate indication of
status for units in different networks with diverse situations like
weight-rescaling operation. Further, we show that feature entropy decreases as
the layer goes deeper and shares almost simultaneous trend with loss during
training. We show that by investigating the feature entropy of units on only
training data, it could give discrimination between networks with different
generalization ability from the view of the effectiveness of feature
representations.
