Guess what? You can boost Federated Learning for free
  Federated Learning (FL) exploits the computation power of edge devices,
typically mobile phones, while addressing privacy by letting data stay where it
is produced. FL has been used by major service providers to improve item
recommendations, virtual keyboards and text auto-completion services. While
appealing, FL performance is hampered by multiple factors: i) differing
capabilities of participating clients (e.g., computing power, memory and
network connectivity); ii) strict training constraints where devices must be
idle, plugged-in and connected to an unmetered WiFi; and iii) data
heterogeneity (a.k.a non-IIDness). Together, these lead to uneven
participation, straggling, dropout and consequently slow down convergence,
challenging the practicality of FL for many applications.
  In this paper, we present GeL, the Guess and Learn algorithm, that
significantly speeds up convergence by guessing model updates for each client.
The power of GeL is to effectively perform ''free'' learning steps without any
additional gradient computations. GeL provides these guesses through clever use
of moments in the Adam optimizer in combination with the last computed gradient
on clients. Our extensive experimental study involving five standard FL
benchmarks shows that GeL speeds up the convergence up to 1.64x in
heterogeneous systems in the presence of data non-IIDness, saving tens of
thousands of gradient computations.
