Learning Theory Can (Sometimes) Explain Generalisation in Graph Neural
  Networks
  In recent years, several results in the supervised learning setting suggested
that classical statistical learning-theoretic measures, such as VC dimension,
do not adequately explain the performance of deep learning models which
prompted a slew of work in the infinite-width and iteration regimes. However,
there is little theoretical explanation for the success of neural networks
beyond the supervised setting. In this paper we argue that, under some
distributional assumptions, classical learning-theoretic measures can
sufficiently explain generalization for graph neural networks in the
transductive setting. In particular, we provide a rigorous analysis of the
performance of neural networks in the context of transductive inference,
specifically by analysing the generalisation properties of graph convolutional
networks for the problem of node classification. While VC Dimension does result
in trivial generalisation error bounds in this setting as well, we show that
transductive Rademacher complexity can explain the generalisation properties of
graph convolutional networks for stochastic block models. We further use the
generalisation error bounds based on transductive Rademacher complexity to
demonstrate the role of graph convolutions and network architectures in
achieving smaller generalisation error and provide insights into when the graph
structure can help in learning. The findings of this paper could re-new the
interest in studying generalisation in neural networks in terms of
learning-theoretic measures, albeit in specific problems.
