Stochastic Coded Federated Learning with Convergence and Privacy
  Guarantees
  Federated learning (FL) has attracted much attention as a privacy-preserving
distributed machine learning framework, where many clients collaboratively
train a machine learning model by exchanging model updates with a parameter
server instead of sharing their raw data. Nevertheless, FL training suffers
from slow convergence and unstable performance due to stragglers caused by the
heterogeneous computational resources of clients and fluctuating communication
rates. This paper proposes a coded FL framework to mitigate the straggler
issue, namely stochastic coded federated learning (SCFL). In this framework,
each client generates a privacy-preserving coded dataset by adding additive
noise to the random linear combination of its local data. The server collects
the coded datasets from all the clients to construct a composite dataset, which
helps to compensate for the straggling effect. In the training process, the
server as well as clients perform mini-batch stochastic gradient descent (SGD),
and the server adds a make-up term in model aggregation to obtain unbiased
gradient estimates. We characterize the privacy guarantee by the mutual
information differential privacy (MI-DP) and analyze the convergence
performance in federated learning. Besides, we demonstrate a
privacy-performance tradeoff of the proposed SCFL method by analyzing the
influence of the privacy constraint on the convergence rate. Finally, numerical
experiments corroborate our analysis and show the benefits of SCFL in achieving
fast convergence while preserving data privacy.
