Improving Attention Mechanism in Graph Neural Networks via Cardinality
  Preservation
  Graph Neural Networks (GNNs) are powerful to learn the representation of
graph-structured data. Most of the GNNs use the message-passing scheme, where
the embedding of a node is iteratively updated by aggregating the information
of its neighbors. To achieve a better expressive capability of node influences,
attention mechanism has grown to be popular to assign trainable weights to the
nodes in aggregation. Though the attention-based GNNs have achieved remarkable
results in various tasks, a clear understanding of their discriminative
capacities is missing. In this work, we present a theoretical analysis of the
representational properties of the GNN that adopts the attention mechanism as
an aggregator. Our analysis determines all cases when those attention-based
GNNs can always fail to distinguish certain distinct structures. Those cases
appear due to the ignorance of cardinality information in attention-based
aggregation. To improve the performance of attention-based GNNs, we propose
cardinality preserved attention (CPA) models that can be applied to any kind of
attention mechanisms. Our experiments on node and graph classification confirm
our theoretical analysis and show the competitive performance of our CPA
models.
