FLIX: A Simple and Communication-Efficient Alternative to Local Methods
  in Federated Learning
  Federated Learning (FL) is an increasingly popular machine learning paradigm
in which multiple nodes try to collaboratively learn under privacy,
communication and multiple heterogeneity constraints. A persistent problem in
federated learning is that it is not clear what the optimization objective
should be: the standard average risk minimization of supervised learning is
inadequate in handling several major constraints specific to federated
learning, such as communication adaptivity and personalization control. We
identify several key desiderata in frameworks for federated learning and
introduce a new framework, FLIX, that takes into account the unique challenges
brought by federated learning. FLIX has a standard finite-sum form, which
enables practitioners to tap into the immense wealth of existing (potentially
non-local) methods for distributed optimization. Through a smart initialization
that does not require any communication, FLIX does not require the use of local
steps but is still provably capable of performing dissimilarity regularization
on par with local methods. We give several algorithms for solving the FLIX
formulation efficiently under communication constraints. Finally, we
corroborate our theoretical results with extensive experimentation.
