Gradual Federated Learning with Simulated Annealing
  Federated averaging (FedAvg) is a popular federated learning (FL) technique
that updates the global model by averaging local models and then transmits the
updated global model to devices for their local model update. One main
limitation of FedAvg is that the average-based global model is not necessarily
better than local models in the early stage of the training process so that
FedAvg might diverge in realistic scenarios, especially when the data is
non-identically distributed across devices and the number of data samples
varies significantly from device to device. In this paper, we propose a new FL
technique based on simulated annealing. The key idea of the proposed technique,
henceforth referred to as \textit{simulated annealing-based FL} (SAFL), is to
allow a device to choose its local model when the global model is immature.
Specifically, by exploiting the simulated annealing strategy, we make each
device choose its local model with high probability in early iterations when
the global model is immature. From extensive numerical experiments using
various benchmark datasets, we demonstrate that SAFL outperforms the
conventional FedAvg technique in terms of the convergence speed and the
classification accuracy.
