Robust Aggregation for Federated Learning
  Federated learning is the centralized training of statistical models from
decentralized data on mobile devices while preserving the privacy of each
device. We present a robust aggregation approach to make federated learning
robust to settings when a fraction of the devices may be sending corrupted
updates to the server. The approach relies on a robust aggregation oracle based
on the geometric median, which returns a robust aggregate using a constant
number of iterations of a regular non-robust averaging oracle. The robust
aggregation oracle is privacy-preserving, similar to the non-robust secure
average oracle it builds upon. We establish its convergence for least squares
estimation of additive models. We provide experimental results with linear
models and deep networks for three tasks in computer vision and natural
language processing. The robust aggregation approach is agnostic to the level
of corruption; it outperforms the classical aggregation approach in terms of
robustness when the level of corruption is high, while being competitive in the
regime of low corruption. Two variants, a faster one with one-step robust
aggregation and another one with on-device personalization, round off the
paper.
