Minimax Multi-Task Learning and a Generalized Loss-Compositional
  Paradigm for MTL
  Since its inception, the modus operandi of multi-task learning (MTL) has been
to minimize the task-wise mean of the empirical risks. We introduce a
generalized loss-compositional paradigm for MTL that includes a spectrum of
formulations as a subfamily. One endpoint of this spectrum is minimax MTL: a
new MTL formulation that minimizes the maximum of the tasks' empirical risks.
Via a certain relaxation of minimax MTL, we obtain a continuum of MTL
formulations spanning minimax MTL and classical MTL. The full paradigm itself
is loss-compositional, operating on the vector of empirical risks. It
incorporates minimax MTL, its relaxations, and many new MTL formulations as
special cases. We show theoretically that minimax MTL tends to avoid worst case
outcomes on newly drawn test tasks in the learning to learn (LTL) test setting.
The results of several MTL formulations on synthetic and real problems in the
MTL and LTL test settings are encouraging.
