CoFED: Cross-Silo Heterogeneous Federated Multitask Learning via
  Cotraining
  Federated learning (FL) is a machine learning technique that enables
participants to collaboratively train high-quality models without exchanging
their private data. Participants utilizing cross-silo FL (CS-FL) settings are
independent organizations with different task needs, and they are concerned not
only with data privacy but also with independently training their unique models
due to intellectual property considerations. Most existing FL schemes are
incapable of satisfying the above scenarios. In this paper, we propose a
communication-efficient FL (CoFED) scheme based on the pseudolabeling of
unlabeled data via a process such as cotraining. To the best of our knowledge,
this is the first FL scheme that is simultaneously compatible with
heterogeneous tasks, heterogeneous models, and heterogeneous training
algorithms. Experimental results show that CoFED achieves better performance
than competing approaches with a lower communication cost. This is especially
true for non-independent and identically distributed (IID) settings and
heterogeneous models, where the proposed method achieves a 35% performance
improvement.
