FedDR -- Randomized Douglas-Rachford Splitting Algorithms for Nonconvex
  Federated Composite Optimization
  We develop two new algorithms, called, FedDR and asyncFedDR, for solving a
fundamental nonconvex composite optimization problem in federated learning. Our
algorithms rely on a novel combination between a nonconvex Douglas-Rachford
splitting method, randomized block-coordinate strategies, and asynchronous
implementation. They can also handle convex regularizers. Unlike recent methods
in the literature, e.g., FedSplit and FedPD, our algorithms update only a
subset of users at each communication round, and possibly in an asynchronous
manner, making them more practical. These new algorithms can handle statistical
and system heterogeneity, which are the two main challenges in federated
learning, while achieving the best known communication complexity. In fact, our
new algorithms match the communication complexity lower bound up to a constant
factor under standard assumptions. Our numerical experiments illustrate the
advantages of our methods over existing algorithms on synthetic and real
datasets.
