Streaming Graph Neural Networks via Continual Learning
  Graph neural networks (GNNs) have achieved strong performance in various
applications. In the real world, network data is usually formed in a streaming
fashion. The distributions of patterns that refer to neighborhood information
of nodes may shift over time. The GNN model needs to learn the new patterns
that cannot yet be captured. But learning incrementally leads to the
catastrophic forgetting problem that historical knowledge is overwritten by
newly learned knowledge. Therefore, it is important to train GNN model to learn
new patterns and maintain existing patterns simultaneously, which few works
focus on. In this paper, we propose a streaming GNN model based on continual
learning so that the model is trained incrementally and up-to-date node
representations can be obtained at each time step. Firstly, we design an
approximation algorithm to detect new coming patterns efficiently based on
information propagation. Secondly, we combine two perspectives of data
replaying and model regularization for existing pattern consolidation.
Specially, a hierarchy-importance sampling strategy for nodes is designed and a
weighted regularization term for GNN parameters is derived, achieving greater
stability and generalization of knowledge consolidation. Our model is evaluated
on real and synthetic data sets and compared with multiple baselines. The
results of node classification prove that our model can efficiently update
model parameters and achieve comparable performance to model retraining. In
addition, we also conduct a case study on the synthetic data, and carry out
some specific analysis for each part of our model, illustrating its ability to
learn new knowledge and maintain existing knowledge from different
perspectives.
