LoSAC: An Efficient Local Stochastic Average Control Method for
  Federated Optimization
  Federated optimization (FedOpt), which targets at collaboratively training a
learning model across a large number of distributed clients, is vital for
federated learning. The primary concerns in FedOpt can be attributed to the
model divergence and communication efficiency, which significantly affect the
performance. In this paper, we propose a new method, i.e., LoSAC, to learn from
heterogeneous distributed data more efficiently. Its key algorithmic insight is
to locally update the estimate for the global full gradient after {each}
regular local model update. Thus, LoSAC can keep clients' information refreshed
in a more compact way. In particular, we have studied the convergence result
for LoSAC. Besides, the bonus of LoSAC is the ability to defend the information
leakage from the recent technique Deep Leakage Gradients (DLG). Finally,
experiments have verified the superiority of LoSAC comparing with
state-of-the-art FedOpt algorithms. Specifically, LoSAC significantly improves
communication efficiency by more than $100\%$ on average, mitigates the model
divergence problem and equips with the defense ability against DLG.
