Byzantine-Robust Decentralized Learning via Self-Centered Clipping
  In this paper, we study the challenging task of Byzantine-robust
decentralized training on arbitrary communication graphs. Unlike federated
learning where workers communicate through a server, workers in the
decentralized environment can only talk to their neighbors, making it harder to
reach consensus. We identify a novel dissensus attack in which few malicious
nodes can take advantage of information bottlenecks in the topology to poison
the collaboration. To address these issues, we propose a Self-Centered Clipping
(SCClip) algorithm for Byzantine-robust consensus and optimization, which is
the first to provably converge to a $O(\delta_{\max}\zeta^2/\gamma^2)$
neighborhood of the stationary point for non-convex objectives under standard
assumptions. Finally, we demonstrate the encouraging empirical performance of
SCClip under a large number of attacks.
