Agnostic Federated Learning
  A key learning scenario in large-scale applications is that of federated
learning, where a centralized model is trained based on data originating from a
large number of clients. We argue that, with the existing training and
inference, federated models can be biased towards different clients. Instead,
we propose a new framework of agnostic federated learning, where the
centralized model is optimized for any target distribution formed by a mixture
of the client distributions. We further show that this framework naturally
yields a notion of fairness. We present data-dependent Rademacher complexity
guarantees for learning with this objective, which guide the definition of an
algorithm for agnostic federated learning. We also give a fast stochastic
optimization algorithm for solving the corresponding optimization problem, for
which we prove convergence bounds, assuming a convex loss function and
hypothesis set. We further empirically demonstrate the benefits of our approach
in several datasets. Beyond federated learning, our framework and algorithm can
be of interest to other learning scenarios such as cloud computing, domain
adaptation, drifting, and other contexts where the training and test
distributions do not coincide.
