GraphFM: Improving Large-Scale GNN Training via Feature Momentum
  Training of graph neural networks (GNNs) for large-scale node classification
is challenging. A key difficulty lies in obtaining accurate hidden node
representations while avoiding the neighborhood explosion problem. Here, we
propose a new technique, named as feature momentum (FM), that uses a momentum
step to incorporate historical embeddings when updating feature
representations. We develop two specific algorithms, known as GraphFM-IB and
GraphFM-OB, that consider in-batch and out-of-batch data, respectively.
GraphFM-IB applies FM to in-batch sampled data, while GraphFM-OB applies FM to
out-of-batch data that are 1-hop neighborhood of in-batch data. We provide a
rigorous convergence analysis for GraphFM-IB and theoretical insight of
GraphFM-OB for the estimation error of feature embeddings. Empirically, we
observe that GraphFM-IB can effectively alleviate the neighborhood explosion
problem of existing methods. In addition, GraphFM-OB achieves promising
performance on multiple large-scale graph datasets.
