NodePiece: Compositional and Parameter-Efficient Representations of
  Large Knowledge Graphs
  Conventional representation learning algorithms for knowledge graphs (KG) map
each entity to a unique embedding vector. Such a shallow lookup results in a
linear growth of memory consumption for storing the embedding matrix and incurs
high computational costs when working with real-world KGs. Drawing parallels
with subword tokenization commonly used in NLP, we explore the landscape of
more parameter-efficient node embedding strategies with possibly sublinear
memory requirements. To this end, we propose NodePiece, an anchor-based
approach to learn a fixed-size entity vocabulary. In NodePiece, a vocabulary of
subword/sub-entity units is constructed from anchor nodes in a graph with known
relation types. Given such a fixed-size vocabulary, it is possible to bootstrap
an encoding and embedding for any entity, including those unseen during
training. Experiments show that NodePiece performs competitively in node
classification, link prediction, and relation prediction tasks while retaining
less than 10% of explicit nodes in a graph as anchors and often having 10x
fewer parameters. To this end, we show that a NodePiece-enabled model
outperforms existing shallow models on a large OGB WikiKG 2 graph having 70x
fewer parameters.
