Fast Learning of Graph Neural Networks with Guaranteed Generalizability:
  One-hidden-layer Case
  Although graph neural networks (GNNs) have made great progress recently on
learning from graph-structured data in practice, their theoretical guarantee on
generalizability remains elusive in the literature. In this paper, we provide a
theoretically-grounded generalizability analysis of GNNs with one hidden layer
for both regression and binary classification problems. Under the assumption
that there exists a ground-truth GNN model (with zero generalization error),
the objective of GNN learning is to estimate the ground-truth GNN parameters
from the training data. To achieve this objective, we propose a learning
algorithm that is built on tensor initialization and accelerated gradient
descent. We then show that the proposed learning algorithm converges to the
ground-truth GNN model for the regression problem, and to a model sufficiently
close to the ground-truth for the binary classification problem. Moreover, for
both cases, the convergence rate of the proposed learning algorithm is proven
to be linear and faster than the vanilla gradient descent algorithm. We further
explore the relationship between the sample complexity of GNNs and their
underlying graph properties. Lastly, we provide numerical experiments to
demonstrate the validity of our analysis and the effectiveness of the proposed
learning algorithm for GNNs.
