Communication-Efficient Federated Learning with Binary Neural Networks
  Federated learning (FL) is a privacy-preserving machine learning setting that
enables many devices to jointly train a shared global model without the need to
reveal their data to a central server. However, FL involves a frequent exchange
of the parameters between all the clients and the server that coordinates the
training. This introduces extensive communication overhead, which can be a
major bottleneck in FL with limited communication links. In this paper, we
consider training the binary neural networks (BNN) in the FL setting instead of
the typical real-valued neural networks to fulfill the stringent delay and
efficiency requirement in wireless edge networks. We introduce a novel FL
framework of training BNN, where the clients only upload the binary parameters
to the server. We also propose a novel parameter updating scheme based on the
Maximum Likelihood (ML) estimation that preserves the performance of the BNN
even without the availability of aggregated real-valued auxiliary parameters
that are usually needed during the training of the BNN. Moreover, for the first
time in the literature, we theoretically derive the conditions under which the
training of BNN is converging. { Numerical results show that the proposed FL
framework significantly reduces the communication cost compared to the
conventional neural networks with typical real-valued parameters, and the
performance loss incurred by the binarization can be further compensated by a
hybrid method.
