Model-Protected Multi-Task Learning
  Multi-task learning (MTL) refers to the paradigm of learning multiple related
tasks together. In contrast, in single-task learning (STL) each individual task
is learned independently. MTL often leads to better trained models because they
can leverage the commonalities among related tasks. However, because MTL
algorithms can ``leak" information from different models across different
tasks, MTL poses a potential security risk. Specifically, an adversary may
participate in the MTL process through one task and thereby acquire the model
information for another task. The previously proposed privacy-preserving MTL
methods protect data instances rather than models, and some of them may
underperform in comparison with STL methods. In this paper, we propose a
privacy-preserving MTL framework to prevent information from each model leaking
to other models based on a perturbation of the covariance matrix of the model
matrix. We study two popular MTL approaches for instantiation, namely, learning
the low-rank and group-sparse patterns of the model matrix. Our algorithms can
be guaranteed not to underperform compared with STL methods. We build our
methods based upon tools for differential privacy, and privacy guarantees,
utility bounds are provided, and heterogeneous privacy budgets are considered.
The experiments demonstrate that our algorithms outperform the baseline methods
constructed by existing privacy-preserving MTL methods on the proposed
model-protection problem.
