FedDQ: Communication-Efficient Federated Learning with Descending
  Quantization
  Federated learning (FL) is an emerging privacy-preserving distributed
learning scheme. The large model size and frequent model aggregation cause
serious communication issue for FL. Many techniques, such as model compression
and quantization, have been proposed to reduce the communication volume.
Existing adaptive quantization schemes use ascending-trend quantization where
the quantization level increases with the training stages. In this paper, we
investigate the adaptive quantization scheme by maximizing the convergence
rate. It is shown that the optimal quantization level is directly related to
the range of the model updates. Given the model is supposed to converge with
training, the range of model updates will get smaller, indicating that the
quantization level should decrease with the training stages. Based on the
theoretical analysis, a descending quantization scheme is thus proposed.
Experimental results show that the proposed descending quantization scheme not
only reduces more communication volume but also helps FL converge faster, when
compared with current ascending quantization scheme.
