Graph Contrastive Learning Automated
  Self-supervised learning on graph-structured data has drawn recent interest
for learning generalizable, transferable and robust representations from
unlabeled graphs. Among many, graph contrastive learning (GraphCL) has emerged
with promising representation learning performance. Unfortunately, unlike its
counterpart on image data, the effectiveness of GraphCL hinges on ad-hoc data
augmentations, which have to be manually picked per dataset, by either rules of
thumb or trial-and-errors, owing to the diverse nature of graph data. That
significantly limits the more general applicability of GraphCL. Aiming to fill
in this crucial gap, this paper proposes a unified bi-level optimization
framework to automatically, adaptively and dynamically select data
augmentations when performing GraphCL on specific graph data. The general
framework, dubbed JOint Augmentation Optimization (JOAO), is instantiated as
min-max optimization. The selections of augmentations made by JOAO are shown to
be in general aligned with previous "best practices" observed from handcrafted
tuning: yet now being automated, more flexible and versatile. Moreover, we
propose a new augmentation-aware projection head mechanism, which will route
output features through different projection heads corresponding to different
augmentations chosen at each training step. Extensive experiments demonstrate
that JOAO performs on par with or sometimes better than the state-of-the-art
competitors including GraphCL, on multiple graph datasets of various scales and
types, yet without resorting to any laborious dataset-specific tuning on
augmentation selection. We release the code at
https://github.com/Shen-Lab/GraphCL_Automated.
