Variance-Reduced Heterogeneous Federated Learning via Stratified Client
  Selection
  Client selection strategies are widely adopted to handle the
communication-efficient problem in recent studies of Federated Learning (FL).
However, due to the large variance of the selected subset's update, prior
selection approaches with a limited sampling ratio cannot perform well on
convergence and accuracy in heterogeneous FL. To address this problem, in this
paper, we propose a novel stratified client selection scheme to reduce the
variance for the pursuit of better convergence and higher accuracy.
Specifically, to mitigate the impact of heterogeneity, we develop
stratification based on clients' local data distribution to derive approximate
homogeneous strata for better selection in each stratum. Concentrating on a
limited sampling ratio scenario, we next present an optimized sample size
allocation scheme by considering the diversity of stratum's variability, with
the promise of further variance reduction. Theoretically, we elaborate the
explicit relation among different selection schemes with regard to variance,
under heterogeneous settings, we demonstrate the effectiveness of our selection
scheme. Experimental results confirm that our approach not only allows for
better performance relative to state-of-the-art methods but also is compatible
with prevalent FL algorithms.
