SignGuard: Byzantine-robust Federated Learning through Collaborative
  Malicious Gradient Filtering
  Gradient-based training in federated learning is known to be vulnerable to
faulty/malicious worker nodes, which are often modeled as Byzantine clients.
Previous work either makes use of auxiliary data at parameter server to verify
the received gradients or leverages statistic-based methods to identify and
remove malicious gradients from Byzantine clients. In this paper, we
acknowledge that auxiliary data may not always be available in practice and
focus on the statistic-based approach. However, recent work on model poisoning
attacks have shown that well-crafted attacks can circumvent most of existing
median- and distance-based statistical defense methods, making malicious
gradients indistinguishable from honest ones. To tackle this challenge, we show
that the element-wise sign of gradient vector can provide valuable insight in
detecting model poisoning attacks. Based on our theoretical analysis of
state-of-the-art attack, we propose a novel approach, \textit{SignGuard}, to
enable Byzantine-robust federated learning through collaborative malicious
gradient filtering. More precisely, the received gradients are first processed
to generate relevant magnitude, sign, and similarity statistics, which are then
collaboratively utilized by multiple, parallel filters to eliminate malicious
gradients before final aggregation. We further provide theoretical analysis of
SignGuard by quantifying its convergence with appropriate choice of learning
rate and under non-IID training data. Finally, extensive experiments of image
and text classification tasks - including MNIST, Fashion-MNIST, CIFAR-10, and
AG-News - are conducted together with recently proposed attacks and defense
strategies. The numerical results demonstrate the effectiveness and superiority
of our proposed approach.
