Scalable Graph Neural Network Training: The Case for Sampling
  Graph Neural Networks (GNNs) are a new and increasingly popular family of
deep neural network architectures to perform learning on graphs. Training them
efficiently is challenging due to the irregular nature of graph data. The
problem becomes even more challenging when scaling to large graphs that exceed
the capacity of single devices. Standard approaches to distributed DNN
training, such as data and model parallelism, do not directly apply to GNNs.
Instead, two different approaches have emerged in the literature: whole-graph
and sample-based training.
  In this paper, we review and compare the two approaches. Scalability is
challenging with both approaches, but we make a case that research should focus
on sample-based training since it is a more promising approach. Finally, we
review recent systems supporting sample-based training.
