Device Heterogeneity in Federated Learning: A Superquantile Approach
  We propose a federated learning framework to handle heterogeneous client
devices which do not conform to the population data distribution. The approach
hinges upon a parameterized superquantile-based objective, where the parameter
ranges over levels of conformity. We present an optimization algorithm and
establish its convergence to a stationary point. We show how to practically
implement it using secure aggregation by interleaving iterations of the usual
federated averaging method with device filtering. We conclude with numerical
experiments on neural networks as well as linear models on tasks from computer
vision and natural language processing.
