Compression Boosts Differentially Private Federated Learning
  Federated Learning allows distributed entities to train a common model
collaboratively without sharing their own data. Although it prevents data
collection and aggregation by exchanging only parameter updates, it remains
vulnerable to various inference and reconstruction attacks where a malicious
entity can learn private information about the participants' training data from
the captured gradients. Differential Privacy is used to obtain theoretically
sound privacy guarantees against such inference attacks by noising the
exchanged update vectors. However, the added noise is proportional to the model
size which can be very large with modern neural networks. This can result in
poor model quality. In this paper, compressive sensing is used to reduce the
model size and hence increase model quality without sacrificing privacy. We
show experimentally, using 2 datasets, that our privacy-preserving proposal can
reduce the communication costs by up to 95% with only a negligible performance
penalty compared to traditional non-private federated learning schemes.
