Dorylus: Affordable, Scalable, and Accurate GNN Training with
  Distributed CPU Servers and Serverless Threads
  A graph neural network (GNN) enables deep learning on structured graph data.
There are two major GNN training obstacles: 1) it relies on high-end servers
with many GPUs which are expensive to purchase and maintain, and 2) limited
memory on GPUs cannot scale to today's billion-edge graphs. This paper presents
Dorylus: a distributed system for training GNNs. Uniquely, Dorylus can take
advantage of serverless computing to increase scalability at a low cost.
  The key insight guiding our design is computation separation. Computation
separation makes it possible to construct a deep, bounded-asynchronous pipeline
where graph and tensor parallel tasks can fully overlap, effectively hiding the
network latency incurred by Lambdas. With the help of thousands of Lambda
threads, Dorylus scales GNN training to billion-edge graphs. Currently, for
large graphs, CPU servers offer the best performance-per-dollar over GPU
servers. Just using Lambdas on top of CPU servers offers up to 2.75x more
performance-per-dollar than training only with CPU servers. Concretely, Dorylus
is 1.22x faster and 4.83x cheaper than GPU servers for massive sparse graphs.
Dorylus is up to 3.8x faster and 10.7x cheaper compared to existing
sampling-based systems.
