Collaboration Equilibrium in Federated Learning
  Federated learning (FL) refers to the paradigm of learning models over a
collaborative research network involving multiple clients without sacrificing
privacy. Recently, there have been rising concerns on the distributional
discrepancies across different clients, which could even cause
counterproductive consequences when collaborating with others. While it is not
necessarily that collaborating with all clients will achieve the best
performance, in this paper, we study a rational collaboration called
``collaboration equilibrium'' (CE), where smaller collaboration coalitions are
formed. Each client collaborates with certain members who maximally improve the
model learning and isolates the others who make little contribution. We propose
the concept of benefit graph which describes how each client can benefit from
collaborating with other clients and advance a Pareto optimization approach to
identify the optimal collaborators. Then we theoretically prove that we can
reach a CE from the benefit graph through an iterative graph operation. Our
framework provides a new way of setting up collaborations in a research
network. Experiments on both synthetic and real world data sets are provided to
demonstrate the effectiveness of our method.
