Where is the Information in a Deep Neural Network?
  Whatever information a deep neural network has gleaned from training data is
encoded in its weights. How this information affects the response of the
network to future data remains largely an open question. Indeed, even defining
and measuring information entails some subtleties, since a trained network is a
deterministic map, so standard information measures can be degenerate. We
measure information in a neural network via the optimal trade-off between
accuracy of the response and complexity of the weights, measured by their
coding length. Depending on the choice of code, the definition can reduce to
standard measures such as Shannon Mutual Information and Fisher Information.
However, the more general definition allows us to relate information to
generalization and invariance, through a novel notion of effective information
in the activations of a deep network. We establish a novel relation between the
information in the weights and the effective information in the activations,
and use this result to show that models with low (information) complexity not
only generalize better, but are bound to learn invariant representations of
future inputs. These relations hinge not only on the architecture of the model,
but also on how it is trained, highlighting the complex inter-dependency
between the class of functions implemented by deep neural networks, the loss
function used for training them from finite data, and the inductive bias
implicit in the optimization.
