Graph Information Vanishing Phenomenon inImplicit Graph Neural Networks
  One of the key problems of GNNs is how to describe the importance of neighbor
nodes in the aggregation process for learning node representations. A class of
GNNs solves this problem by learning implicit weights to represent the
importance of neighbor nodes, which we call implicit GNNs such as Graph
Attention Network. The basic idea of implicit GNNs is to introduce graph
information with special properties followed by Learnable Transformation
Structures (LTS) which encode the importance of neighbor nodes via a
data-driven way. In this paper, we argue that LTS makes the special properties
of graph information disappear during the learning process, resulting in graph
information unhelpful for learning node representations. We call this
phenomenon Graph Information Vanishing (GIV). Also, we find that LTS maps
different graph information into highly similar results. To validate the above
two points, we design two sets of 70 random experiments on five Implicit GNNs
methods and seven benchmark datasets by using a random permutation operator to
randomly disrupt the order of graph information and replacing graph information
with random values. We find that randomization does not affect the model
performance in 93\% of the cases, with about 7 percentage causing an average
0.5\% accuracy loss. And the cosine similarity of output results, generated by
LTS mapping different graph information, over 99\% with an 81\% proportion. The
experimental results provide evidence to support the existence of GIV in
Implicit GNNs and imply that the existing methods of Implicit GNNs do not make
good use of graph information. The relationship between graph information and
LTS should be rethought to ensure that graph information is used in node
representation.
