Multi-task Supervised Learning via Cross-learning
  In this paper we consider a problem known as multi-task learning, consisting
of fitting a set of classifier or regression functions intended for solving
different tasks. In our novel formulation, we couple the parameters of these
functions, so that they learn in their task specific domains while staying
close to each other. This facilitates cross-fertilization in which data
collected across different domains help improving the learning performance at
each other task. First, we present a simplified case in which the goal is to
estimate the means of two Gaussian variables, for the purpose of gaining some
insights on the advantage of the proposed cross-learning strategy. Then we
provide a stochastic projected gradient algorithm to perform cross-learning
over a generic loss function. If the number of parameters is large, then the
projection step becomes computationally expensive. To avoid this situation, we
derive a primal-dual algorithm that exploits the structure of the dual problem,
achieving a formulation whose complexity only depends on the number of tasks.
Preliminary numerical experiments for image classification by neural networks
trained on a dataset divided in different domains corroborate that the
cross-learned function outperforms both the task-specific and the consensus
approaches.
