Towards Efficient Synchronous Federated Training: A Survey on System
  Optimization Strategies
  The increasing demand for privacy-preserving collaborative learning has given
rise to a new computing paradigm called federated learning (FL), in which
clients collaboratively train a machine learning (ML) model without revealing
their private training data. Given an acceptable level of privacy guarantee,
the goal of FL is to minimize the time-to-accuracy of model training. Compared
with distributed ML in data centers, there are four distinct challenges to
achieving short time-to-accuracy in FL training, namely the lack of information
for optimization, the tradeoff between statistical and system utility, client
heterogeneity, and large configuration space. In this paper, we survey recent
works in addressing these challenges and present them following a typical
training workflow through three phases: client selection, configuration, and
reporting. We also review system works including measurement studies and
benchmarking tools that aim to support FL developers.
