Attack of the Tails: Yes, You Really Can Backdoor Federated Learning
  Due to its decentralized nature, Federated Learning (FL) lends itself to
adversarial attacks in the form of backdoors during training. The goal of a
backdoor is to corrupt the performance of the trained model on specific
sub-tasks (e.g., by classifying green cars as frogs). A range of FL backdoor
attacks have been introduced in the literature, but also methods to defend
against them, and it is currently an open question whether FL systems can be
tailored to be robust against backdoors. In this work, we provide evidence to
the contrary. We first establish that, in the general case, robustness to
backdoors implies model robustness to adversarial examples, a major open
problem in itself. Furthermore, detecting the presence of a backdoor in a FL
model is unlikely assuming first order oracles or polynomial time. We couple
our theoretical results with a new family of backdoor attacks, which we refer
to as edge-case backdoors. An edge-case backdoor forces a model to misclassify
on seemingly easy inputs that are however unlikely to be part of the training,
or test data, i.e., they live on the tail of the input distribution. We explain
how these edge-case backdoors can lead to unsavory failures and may have
serious repercussions on fairness, and exhibit that with careful tuning at the
side of the adversary, one can insert them across a range of machine learning
tasks (e.g., image classification, OCR, text prediction, sentiment analysis).
