Dynamic Sampling and Selective Masking for Communication-Efficient
  Federated Learning
  Federated learning (FL) is a novel machine learning setting that enables
on-device intelligence via decentralized training and federated optimization.
Deep neural networks' rapid development facilitates the learning techniques for
modeling complex problems and emerges into federated deep learning under the
federated setting. However, the tremendous amount of model parameters burdens
the communication network with a high load of transportation. This paper
introduces two approaches for improving communication efficiency by dynamic
sampling and top-$k$ selective masking. The former controls the fraction of
selected client models dynamically, while the latter selects parameters with
top-$k$ largest values of difference for federated updating. Experiments on
convolutional image classification and recurrent language modeling are
conducted on three public datasets to show our proposed methods' effectiveness.
