FedSplit: An algorithmic framework for fast federated optimization
  Motivated by federated learning, we consider the hub-and-spoke model of
distributed optimization in which a central authority coordinates the
computation of a solution among many agents while limiting communication. We
first study some past procedures for federated optimization, and show that
their fixed points need not correspond to stationary points of the original
optimization problem, even in simple convex settings with deterministic
updates. In order to remedy these issues, we introduce FedSplit, a class of
algorithms based on operator splitting procedures for solving distributed
convex minimization with additive structure. We prove that these procedures
have the correct fixed points, corresponding to optima of the original
optimization problem, and we characterize their convergence rates under
different settings. Our theory shows that these methods are provably robust to
inexact computation of intermediate local quantities. We complement our theory
with some simple experiments that demonstrate the benefits of our methods in
practice.
