Generalizing Aggregation Functions in GNNs:High-Capacity GNNs via
  Nonlinear Neighborhood Aggregators
  Graph neural networks (GNNs) have achieved great success in many graph
learning tasks. The main aspect powering existing GNNs is the multi-layer
network architecture to learn the nonlinear graph representations for the
specific learning tasks. The core operation in GNNs is message propagation in
which each node updates its representation by aggregating its neighbors'
representations. Existing GNNs mainly adopt either linear neighborhood
aggregation (mean,sum) or max aggregator in their message propagation. (1) For
linear aggregators, the whole nonlinearity and network's capacity of GNNs are
generally limited due to deeper GNNs usually suffer from over-smoothing issue.
(2) For max aggregator, it usually fails to be aware of the detailed
information of node representations within neighborhood. To overcome these
issues, we re-think the message propagation mechanism in GNNs and aim to
develop the general nonlinear aggregators for neighborhood information
aggregation in GNNs. One main aspect of our proposed nonlinear aggregators is
that they provide the optimally balanced aggregators between max and mean/sum
aggregations. Thus, our aggregators can inherit both (i) high nonlinearity that
increases network's capacity and (ii) detail-sensitivity that preserves the
detailed information of representations together in GNNs' message propagation.
Promising experiments on several datasets show the effectiveness of the
proposed nonlinear aggregators.
