LDP-Fed: Federated Learning with Local Differential Privacy
  This paper presents LDP-Fed, a novel federated learning system with a formal
privacy guarantee using local differential privacy (LDP). Existing LDP
protocols are developed primarily to ensure data privacy in the collection of
single numerical or categorical values, such as click count in Web access logs.
However, in federated learning model parameter updates are collected
iteratively from each participant and consist of high dimensional, continuous
values with high precision (10s of digits after the decimal point), making
existing LDP protocols inapplicable. To address this challenge in LDP-Fed, we
design and develop two novel approaches. First, LDP-Fed's LDP Module provides a
formal differential privacy guarantee for the repeated collection of model
training parameters in the federated training of large-scale neural networks
over multiple individual participants' private datasets. Second, LDP-Fed
implements a suite of selection and filtering techniques for perturbing and
sharing select parameter updates with the parameter server. We validate our
system deployed with a condensed LDP protocol in training deep neural networks
on public data. We compare this version of LDP-Fed, coined CLDP-Fed, with other
state-of-the-art approaches with respect to model accuracy, privacy
preservation, and system capabilities.
