Federated Learning with Heterogeneous Data: A Superquantile Optimization
  Approach
  We present a federated learning framework that is designed to robustly
deliver good predictive performance across individual clients with
heterogeneous data. The proposed approach hinges upon a superquantile-based
learning objective that captures the tail statistics of the error distribution
over heterogeneous clients. We present a stochastic training algorithm which
interleaves differentially private client reweighting steps with federated
averaging steps. The proposed algorithm is supported with finite time
convergence guarantees that cover both convex and non-convex settings.
Experimental results on benchmark datasets for federated learning demonstrate
that our approach is competitive with classical ones in terms of average error
and outperforms them in terms of tail statistics of the error.
