Personalized Federated Learning: A Unified Framework and Universal
  Optimization Techniques
  We study the optimization aspects of personalized Federated Learning (FL). We
propose general optimizers that can be used to solve essentially any existing
personalized FL objective, namely a tailored variant of Local SGD and variants
of accelerated coordinate descent/accelerated SVRCD. By studying a general
personalized objective that is capable of recovering essentially any existing
personalized FL objective as a special case, we develop a universal
optimization theory applicable to all strongly convex personalized FL models in
the literature. We demonstrate the practicality and/or optimality of our
methods both in terms of communication and local computation. Surprisingly
enough, our general optimization solvers and theory are capable of recovering
best-known communication and computation guarantees for solving specific
personalized FL objectives. Thus, our proposed methods can be taken as
universal optimizers that make the design of task-specific optimizers
unnecessary in many cases.
