Federated Learning over Wireless Networks: A Band-limited Coordinated
  Descent Approach
  We consider a many-to-one wireless architecture for federated learning at the
network edge, where multiple edge devices collaboratively train a model using
local data. The unreliable nature of wireless connectivity, together with
constraints in computing resources at edge devices, dictates that the local
updates at edge devices should be carefully crafted and compressed to match the
wireless communication resources available and should work in concert with the
receiver. Thus motivated, we propose SGD-based bandlimited coordinate descent
algorithms for such settings. Specifically, for the wireless edge employing
over-the-air computing, a common subset of k-coordinates of the gradient
updates across edge devices are selected by the receiver in each iteration, and
then transmitted simultaneously over k sub-carriers, each experiencing
time-varying channel conditions. We characterize the impact of communication
error and compression, in terms of the resulting gradient bias and mean squared
error, on the convergence of the proposed algorithms. We then study
learning-driven communication error minimization via joint optimization of
power allocation and learning rates. Our findings reveal that optimal power
allocation across different sub-carriers should take into account both the
gradient values and channel conditions, thus generalizing the widely used
water-filling policy. We also develop sub-optimal distributed solutions
amenable to implementation.
