Communication Efficient Federated Learning over Multiple Access Channels
  In this work, we study the problem of federated learning (FL), where
distributed users aim to jointly train a machine learning model with the help
of a parameter server (PS). In each iteration of FL, users compute local
gradients, followed by transmission of the quantized gradients for subsequent
aggregation and model updates at PS. One of the challenges of FL is that of
communication overhead due to FL's iterative nature and large model sizes. One
recent direction to alleviate communication bottleneck in FL is to let users
communicate simultaneously over a multiple access channel (MAC), possibly
making better use of the communication resources.
  In this paper, we consider the problem of FL learning over a MAC. In
particular, we focus on the design of digital gradient transmission schemes
over a MAC, where gradients at each user are first quantized, and then
transmitted over a MAC to be decoded individually at the PS. When designing
digital FL schemes over MACs, there are new opportunities to assign different
amount of resources (such as rate or bandwidth) to different users based on a)
the informativeness of the gradients at each user, and b) the underlying
channel conditions. We propose a stochastic gradient quantization scheme, where
the quantization parameters are optimized based on the capacity region of the
MAC. We show that such channel aware quantization for FL outperforms uniform
quantization, particularly when users experience different channel conditions,
and when have gradients with varying levels of informativeness.
