A Brief Summary of Interactions Between Meta-Learning and
  Self-Supervised Learning
  This paper briefly reviews the connections between meta-learning and
self-supervised learning. Meta-learning can be applied to improve model
generalization capability and to construct general AI algorithms.
Self-supervised learning utilizes self-supervision from original data and
extracts higher-level generalizable features through unsupervised pre-training
or optimization of contrastive loss objectives. In self-supervised learning,
data augmentation techniques are widely applied and data labels are not
required since pseudo labels can be estimated from trained models on similar
tasks. Meta-learning aims to adapt trained deep models to solve diverse tasks
and to develop general AI algorithms. We review the associations of
meta-learning with both generative and contrastive self-supervised learning
models. Unlabeled data from multiple sources can be jointly considered even
when data sources are vastly different. We show that an integration of
meta-learning and self-supervised learning models can best contribute to the
improvement of model generalization capability. Self-supervised learning guided
by meta-learner and general meta-learning algorithms under self-supervision are
both examples of possible combinations.
