FedRAD: Federated Robust Adaptive Distillation
  The robustness of federated learning (FL) is vital for the distributed
training of an accurate global model that is shared among large number of
clients. The collaborative learning framework by typically aggregating model
updates is vulnerable to model poisoning attacks from adversarial clients.
Since the shared information between the global server and participants are
only limited to model parameters, it is challenging to detect bad model
updates. Moreover, real-world datasets are usually heterogeneous and not
independent and identically distributed (Non-IID) among participants, which
makes the design of such robust FL pipeline more difficult. In this work, we
propose a novel robust aggregation method, Federated Robust Adaptive
Distillation (FedRAD), to detect adversaries and robustly aggregate local
models based on properties of the median statistic, and then performing an
adapted version of ensemble Knowledge Distillation. We run extensive
experiments to evaluate the proposed method against recently published works.
The results show that FedRAD outperforms all other aggregators in the presence
of adversaries, as well as in heterogeneous data distributions.
