The Advantage of Conditional Meta-Learning for Biased Regularization and
  Fine-Tuning
  Biased regularization and fine-tuning are two recent meta-learning
approaches. They have been shown to be effective to tackle distributions of
tasks, in which the tasks' target vectors are all close to a common
meta-parameter vector. However, these methods may perform poorly on
heterogeneous environments of tasks, where the complexity of the tasks'
distribution cannot be captured by a single meta-parameter vector. We address
this limitation by conditional meta-learning, inferring a conditioning function
mapping task's side information into a meta-parameter vector that is
appropriate for that task at hand. We characterize properties of the
environment under which the conditional approach brings a substantial advantage
over standard meta-learning and we highlight examples of environments, such as
those with multiple clusters, satisfying these properties. We then propose a
convex meta-algorithm providing a comparable advantage also in practice.
Numerical experiments confirm our theoretical findings.
