Lifelong Learning in Evolving Graphs with Limited Labeled Data and
  Unseen Class Detection
  Large-scale graph data in the real-world are often dynamic rather than
static. The data are changing with new nodes, edges, and even classes appearing
over time, such as in citation networks and research-and-development
collaboration networks. Graph neural networks (GNNs) have emerged as the
standard method for numerous tasks on graph-structured data. In this work, we
employ a two-step procedure to explore how GNNs can be incrementally adapted to
new unseen graph data. First, we analyze the verge between transductive and
inductive learning on standard benchmark datasets. After inductive pretraining,
we add unlabeled data to the graph and show that the models are stable. Then,
we explore the case of continually adding more and more labeled data, while
considering cases, where not all past instances are annotated with class
labels. Furthermore, we introduce new classes while the graph evolves and
explore methods that automatically detect instances from previously unseen
classes. In order to deal with evolving graphs in a principled way, we propose
a lifelong learning framework for graph data along with an evaluation protocol.
In this framework, we evaluate representative GNN architectures. We observe
that implicit knowledge within model parameters becomes more important when
explicit knowledge, i.e., data from past tasks, is limited. We find that in
open-world node classification, the data from surprisingly few past tasks are
sufficient to reach the performance reached by remembering data from all past
tasks. In the challenging task of unseen class detection, we find that using a
weighted cross-entropy loss is important for stability.
