Enhancing the Privacy of Federated Learning with Sketching
  In response to growing concerns about user privacy, federated learning has
emerged as a promising tool to train statistical models over networks of
devices while keeping data localized. Federated learning methods run training
tasks directly on user devices and do not share the raw user data with third
parties. However, current methods still share model updates, which may contain
private information (e.g., one's weight and height), during the training
process. Existing efforts that aim to improve the privacy of federated learning
make compromises in one or more of the following key areas: performance
(particularly communication cost), accuracy, or privacy. To better optimize
these trade-offs, we propose that \textit{sketching algorithms} have a unique
advantage in that they can provide both privacy and performance benefits while
maintaining accuracy. We evaluate the feasibility of sketching-based federated
learning with a prototype on three representative learning models. Our initial
findings show that it is possible to provide strong privacy guarantees for
federated learning without sacrificing performance or accuracy. Our work
highlights that there exists a fundamental connection between privacy and
communication in distributed settings, and suggests important open problems
surrounding the theoretical understanding, methodology, and system design of
practical, private federated learning.
