Convolutional Neural Networks on Graphs with Chebyshev Approximation,
  Revisited
  Designing spectral convolutional networks is a challenging problem in graph
learning. ChebNet, one of the early attempts, approximates the spectral graph
convolutions using Chebyshev polynomials. GCN simplifies ChebNet by utilizing
only the first two Chebyshev polynomials while still outperforming it on
real-world datasets. GPR-GNN and BernNet demonstrate that the Monomial and
Bernstein bases also outperform the Chebyshev basis in terms of learning the
spectral graph convolutions. Such conclusions are counter-intuitive in the
field of approximation theory, where it is established that the Chebyshev
polynomial achieves the optimum convergent rate for approximating a function.
  In this paper, we revisit the problem of approximating the spectral graph
convolutions with Chebyshev polynomials. We show that ChebNet's inferior
performance is primarily due to illegal coefficients learnt by ChebNet
approximating analytic filter functions, which leads to over-fitting. We then
propose ChebNetII, a new GNN model based on Chebyshev interpolation, which
enhances the original Chebyshev polynomial approximation while reducing the
Runge phenomenon. We conducted an extensive experimental study to demonstrate
that ChebNetII can learn arbitrary graph convolutions and achieve superior
performance in both full- and semi-supervised node classification tasks. Most
notably, we scale ChebNetII to a billion graph ogbn-papers100M, showing that
spectral-based GNNs have superior performance.
