Communication-Efficient Federated Learning with Dual-Side Low-Rank
  Compression
  Federated learning (FL) is a promising and powerful approach for training
deep learning models without sharing the raw data of clients. During the
training process of FL, the central server and distributed clients need to
exchange a vast amount of model information periodically. To address the
challenge of communication-intensive training, we propose a new training
method, referred to as federated learning with dual-side low-rank compression
(FedDLR), where the deep learning model is compressed via low-rank
approximations at both the server and client sides. The proposed FedDLR not
only reduces the communication overhead during the training stage but also
directly generates a compact model to speed up the inference process. We shall
provide convergence analysis, investigate the influence of the key parameters,
and empirically show that FedDLR outperforms the state-of-the-art solutions in
terms of both the communication and computation efficiency.
