Adaptive Transfer Learning on Graph Neural Networks
  Graph neural networks (GNNs) is widely used to learn a powerful
representation of graph-structured data. Recent work demonstrates that
transferring knowledge from self-supervised tasks to downstream tasks could
further improve graph representation. However, there is an inherent gap between
self-supervised tasks and downstream tasks in terms of optimization objective
and training data. Conventional pre-training methods may be not effective
enough on knowledge transfer since they do not make any adaptation for
downstream tasks. To solve such problems, we propose a new transfer learning
paradigm on GNNs which could effectively leverage self-supervised tasks as
auxiliary tasks to help the target task. Our methods would adaptively select
and combine different auxiliary tasks with the target task in the fine-tuning
stage. We design an adaptive auxiliary loss weighting model to learn the
weights of auxiliary tasks by quantifying the consistency between auxiliary
tasks and the target task. In addition, we learn the weighting model through
meta-learning. Our methods can be applied to various transfer learning
approaches, it performs well not only in multi-task learning but also in
pre-training and fine-tuning. Comprehensive experiments on multiple downstream
tasks demonstrate that the proposed methods can effectively combine auxiliary
tasks with the target task and significantly improve the performance compared
to state-of-the-art methods.
