Self-supervised Learning on Graphs: Deep Insights and New Direction
  The success of deep learning notoriously requires larger amounts of costly
annotated data. This has led to the development of self-supervised learning
(SSL) that aims to alleviate this limitation by creating domain specific
pretext tasks on unlabeled data. Simultaneously, there are increasing interests
in generalizing deep learning to the graph domain in the form of graph neural
networks (GNNs). GNNs can naturally utilize unlabeled nodes through the simple
neighborhood aggregation that is unable to thoroughly make use of unlabeled
nodes. Thus, we seek to harness SSL for GNNs to fully exploit the unlabeled
data. Different from data instances in the image and text domains, nodes in
graphs present unique structure information and they are inherently linked
indicating not independent and identically distributed (or i.i.d.). Such
complexity is a double-edged sword for SSL on graphs. On the one hand, it
determines that it is challenging to adopt solutions from the image and text
domains to graphs and dedicated efforts are desired. On the other hand, it
provides rich information that enables us to build SSL from a variety of
perspectives. Thus, in this paper, we first deepen our understandings on when,
why, and which strategies of SSL work with GNNs by empirically studying
numerous basic SSL pretext tasks on graphs. Inspired by deep insights from the
empirical studies, we propose a new direction SelfTask to build advanced
pretext tasks that are able to achieve state-of-the-art performance on various
real-world datasets. The specific experimental settings to reproduce our
results can be found in \url{https://github.com/ChandlerBang/SelfTask-GNN}.
