SAFARI: Sparsity enabled Federated Learning with Limited and Unreliable
  Communications
  Federated learning (FL) enables edge devices to collaboratively learn a model
in a distributed fashion. Many existing researches have focused on improving
communication efficiency of high-dimensional models and addressing bias caused
by local updates. However, most of FL algorithms are either based on reliable
communications or assume fixed and known unreliability characteristics. In
practice, networks could suffer from dynamic channel conditions and
non-deterministic disruptions, with time-varying and unknown characteristics.
To this end, in this paper we propose a sparsity enabled FL framework with both
communication efficiency and bias reduction, termed as SAFARI. It makes novel
use of a similarity among client models to rectify and compensate for bias that
is resulted from unreliable communications. More precisely, sparse learning is
implemented on local clients to mitigate communication overhead, while to cope
with unreliable communications, a similarity-based compensation method is
proposed to provide surrogates for missing model updates. We analyze SAFARI
under bounded dissimilarity and with respect to sparse models. It is
demonstrated that SAFARI under unreliable communications is guaranteed to
converge at the same rate as the standard FedAvg with perfect communications.
Implementations and evaluations on CIFAR-10 dataset validate the effectiveness
of SAFARI by showing that it can achieve the same convergence speed and
accuracy as FedAvg with perfect communications, with up to 80% of the model
weights being pruned and a high percentage of client updates missing in each
round.
