FedSynth: Gradient Compression via Synthetic Data in Federated Learning
  Model compression is important in federated learning (FL) with large models
to reduce communication cost. Prior works have been focusing on sparsification
based compression that could desparately affect the global model accuracy. In
this work, we propose a new scheme for upstream communication where instead of
transmitting the model update, each client learns and transmits a light-weight
synthetic dataset such that using it as the training data, the model performs
similarly well on the real training data. The server will recover the local
model update via the synthetic data and apply standard aggregation. We then
provide a new algorithm FedSynth to learn the synthetic data locally.
Empirically, we find our method is comparable/better than random masking
baselines in all three common federated learning benchmark datasets.
