Interpreting and Unifying Graph Neural Networks with An Optimization
  Framework
  Graph Neural Networks (GNNs) have received considerable attention on
graph-structured data learning for a wide variety of tasks. The well-designed
propagation mechanism which has been demonstrated effective is the most
fundamental part of GNNs. Although most of GNNs basically follow a message
passing manner, litter effort has been made to discover and analyze their
essential relations. In this paper, we establish a surprising connection
between different propagation mechanisms with a unified optimization problem,
showing that despite the proliferation of various GNNs, in fact, their proposed
propagation mechanisms are the optimal solution optimizing a feature fitting
function over a wide class of graph kernels with a graph regularization term.
Our proposed unified optimization framework, summarizing the commonalities
between several of the most representative GNNs, not only provides a
macroscopic view on surveying the relations between different GNNs, but also
further opens up new opportunities for flexibly designing new GNNs. With the
proposed framework, we discover that existing works usually utilize naive graph
convolutional kernels for feature fitting function, and we further develop two
novel objective functions considering adjustable graph kernels showing low-pass
or high-pass filtering capabilities respectively. Moreover, we provide the
convergence proofs and expressive power comparisons for the proposed models.
Extensive experiments on benchmark datasets clearly show that the proposed GNNs
not only outperform the state-of-the-art methods but also have good ability to
alleviate over-smoothing, and further verify the feasibility for designing GNNs
with our unified optimization framework.
