Incorporating Symbolic Domain Knowledge into Graph Neural Networks
  Our interest is in scientific problems with the following characteristics:
(1) Data are naturally represented as graphs; (2) The amount of data available
is typically small; and (3) There is significant domain-knowledge, usually
expressed in some symbolic form. These kinds of problems have been addressed
effectively in the past by Inductive Logic Programming (ILP), by virtue of 2
important characteristics: (a) The use of a representation language that easily
captures the relation encoded in graph-structured data, and (b) The inclusion
of prior information encoded as domain-specific relations, that can alleviate
problems of data scarcity, and construct new relations. Recent advances have
seen the emergence of deep neural networks specifically developed for
graph-structured data (Graph-based Neural Networks, or GNNs). While GNNs have
been shown to be able to handle graph-structured data, less has been done to
investigate the inclusion of domain-knowledge. Here we investigate this aspect
of GNNs empirically by employing an operation we term "vertex-enrichment" and
denote the corresponding GNNs as "VEGNNs". Using over 70 real-world datasets
and substantial amounts of symbolic domain-knowledge, we examine the result of
vertex-enrichment across 5 different variants of GNNs. Our results provide
support for the following: (a) Inclusion of domain-knowledge by
vertex-enrichment can significantly improve the performance of a GNN. That is,
the performance VEGNNs is significantly better than GNNs across all GNN
variants; (b) The inclusion of domain-specific relations constructed using ILP
improves the performance of VEGNNs, across all GNN variants. Taken together,
the results provide evidence that it is possible to incorporate symbolic domain
knowledge into a GNN, and that ILP can play an important role in providing
high-level relationships that are not easily discovered by a GNN.
