Fundamental Limits of Deep Graph Convolutional Networks
  Graph convolutional networks (GCNs) are a widely used method for graph
representation learning. To elucidate the capabilities and limitations of GCNs,
we investigate their power, as a function of their number of layers, to
distinguish between different random graph models (corresponding to different
class-conditional distributions in a classification problem) on the basis of
the embeddings of their sample graphs. In particular, the graph models that we
consider arise from graphons, which are the most general possible
parameterizations of infinite exchangeable graph models and which are the
central objects of study in the theory of dense graph limits. We give a precise
characterization of the set of pairs of graphons that are indistinguishable by
a GCN with nonlinear activation functions coming from a certain broad class if
its depth is at least logarithmic in the size of the sample graph. This
characterization is in terms of a degree profile closeness property. Outside
this class, a very simple GCN architecture suffices for distinguishability. We
then exhibit a concrete, infinite class of graphons arising from stochastic
block models that are well-separated in terms of cut distance and are
indistinguishable by a GCN. These results theoretically match empirical
observations of several prior works. To prove our results, we exploit a
connection to random walks on graphs. Finally, we give empirical results on
synthetic and real graph classification datasets, indicating that
indistinguishable graph distributions arise in practice.
