Spectral Complexity-scaled Generalization Bound of Complex-valued Neural
  Networks
  Complex-valued neural networks (CVNNs) have been widely applied to various
fields, especially signal processing and image recognition. However, few works
focus on the generalization of CVNNs, albeit it is vital to ensure the
performance of CVNNs on unseen data. This paper is the first work that proves a
generalization bound for the complex-valued neural network. The bound scales
with the spectral complexity, the dominant factor of which is the spectral norm
product of weight matrices. Further, our work provides a generalization bound
for CVNNs when training data is sequential, which is also affected by the
spectral complexity. Theoretically, these bounds are derived via Maurey
Sparsification Lemma and Dudley Entropy Integral. Empirically, we conduct
experiments by training complex-valued convolutional neural networks on
different datasets: MNIST, FashionMNIST, CIFAR-10, CIFAR-100, Tiny ImageNet,
and IMDB. Spearman's rank-order correlation coefficients and the corresponding
p values on these datasets give strong proof that the spectral complexity of
the network, measured by the weight matrices spectral norm product, has a
statistically significant correlation with the generalization ability.
