Federated Robustness Propagation: Sharing Adversarial Robustness in
  Federated Learning
  Federated learning (FL) emerges as a popular distributed learning schema that
learns a model from a set of participating users without requiring raw data to
be shared. One major challenge of FL comes from heterogeneity in users, which
may have distributionally different (or non-iid) data and varying computation
resources. Just like in centralized learning, FL users also desire model
robustness against malicious attackers at test time. Whereas adversarial
training (AT) provides a sound solution for centralized learning, extending its
usage for FL users has imposed significant challenges, as many users may have
very limited training data as well as tight computational budgets, to afford
the data-hungry and costly AT. In this paper, we study a novel learning setting
that propagates adversarial robustness from high-resource users that can afford
AT, to those low-resource users that cannot afford it, during the FL process.
We show that existing FL techniques cannot effectively propagate adversarial
robustness among non-iid users, and propose a simple yet effective propagation
approach that transfers robustness through carefully designed
batch-normalization statistics. We demonstrate the rationality and
effectiveness of our method through extensive experiments. Especially, the
proposed method is shown to grant FL remarkable robustness even when only a
small portion of users afford AT during learning. Codes will be published upon
acceptance.
