Robust and Efficient Aggregation for Distributed Learning
  Distributed learning paradigms, such as federated and decentralized learning,
allow for the coordination of models across a collection of agents, and without
the need to exchange raw data. Instead, agents compute model updates locally
based on their available data, and subsequently share the update model with a
parameter server or their peers. This is followed by an aggregation step, which
traditionally takes the form of a (weighted) average. Distributed learning
schemes based on averaging are known to be susceptible to outliers. A single
malicious agent is able to drive an averaging-based distributed learning
algorithm to an arbitrarily poor model. This has motivated the development of
robust aggregation schemes, which are based on variations of the median and
trimmed mean. While such procedures ensure robustness to outliers and malicious
behavior, they come at the cost of significantly reduced sample efficiency.
This means that current robust aggregation schemes require significantly higher
agent participation rates to achieve a given level of performance than their
mean-based counterparts in non-contaminated settings. In this work we remedy
this drawback by developing statistically efficient and robust aggregation
schemes for distributed learning.
