Understanding Generalization in Deep Learning via Tensor Methods
  Deep neural networks generalize well on unseen data though the number of
parameters often far exceeds the number of training examples. Recently proposed
complexity measures have provided insights to understanding the
generalizability in neural networks from perspectives of PAC-Bayes, robustness,
overparametrization, compression and so on. In this work, we advance the
understanding of the relations between the network's architecture and its
generalizability from the compression perspective. Using tensor analysis, we
propose a series of intuitive, data-dependent and easily-measurable properties
that tightly characterize the compressibility and generalizability of neural
networks; thus, in practice, our generalization bound outperforms the previous
compression-based ones, especially for neural networks using tensors as their
weight kernels (e.g. CNNs). Moreover, these intuitive measurements provide
further insights into designing neural network architectures with properties
favorable for better/guaranteed generalizability. Our experimental results
demonstrate that through the proposed measurable properties, our generalization
error bound matches the trend of the test error well. Our theoretical analysis
further provides justifications for the empirical success and limitations of
some widely-used tensor-based compression approaches. We also discover the
improvements to the compressibility and robustness of current neural networks
when incorporating tensor operations via our proposed layer-wise structure.
