Adaptive Client Sampling in Federated Learning via Online Learning with
  Bandit Feedback
  Due to the high cost of communication, federated learning (FL) systems need
to sample a subset of clients that are involved in each round of training. As a
result, client sampling plays an important role in FL systems as it affects the
convergence rate of optimization algorithms used to train machine learning
models. Despite its importance, there is limited work on how to sample clients
effectively. In this paper, we cast client sampling as an online learning task
with bandit feedback, which we solve with an online stochastic mirror descent
(OSMD) algorithm designed to minimize the sampling variance. To handle the
tuning parameters in OSMD that depend on the unknown problem parameters, we use
the online ensemble method and doubling trick. We prove a dynamic regret bound
relative to the theoretically optimal sampling sequence. The regret bound
depends on the total variation of the comparator sequence, which naturally
captures the intrinsic difficulty of the problem. To the best of our knowledge,
these theoretical contributions are new and the proof technique is of
independent interest. Through both synthetic and real data experiments, we
illustrate advantages of the proposed client sampling algorithm over the widely
used uniform sampling and existing online learning based sampling strategies.
The proposed adaptive sampling procedure is applicable beyond the FL problem
studied here and can be used to improve the performance of stochastic
optimization procedures such as stochastic gradient descent and stochastic
coordinate descent.
