Breaking the Expressive Bottlenecks of Graph Neural Networks
  Recently, the Weisfeiler-Lehman (WL) graph isomorphism test was used to
measure the expressiveness of graph neural networks (GNNs), showing that the
neighborhood aggregation GNNs were at most as powerful as 1-WL test in
distinguishing graph structures. There were also improvements proposed in
analogy to $k$-WL test ($k>1$). However, the aggregators in these GNNs are far
from injective as required by the WL test, and suffer from weak distinguishing
strength, making it become expressive bottlenecks. In this paper, we improve
the expressiveness by exploring powerful aggregators. We reformulate
aggregation with the corresponding aggregation coefficient matrix, and then
systematically analyze the requirements of the aggregation coefficient matrix
for building more powerful aggregators and even injective aggregators. It can
also be viewed as the strategy for preserving the rank of hidden features, and
implies that basic aggregators correspond to a special case of low-rank
transformations. We also show the necessity of applying nonlinear units ahead
of aggregation, which is different from most aggregation-based GNNs. Based on
our theoretical analysis, we develop two GNN layers, ExpandingConv and
CombConv. Experimental results show that our models significantly boost
performance, especially for large and densely connected graphs.
