LDP-FL: Practical Private Aggregation in Federated Learning with Local
  Differential Privacy
  Train machine learning models on sensitive user data has raised increasing
privacy concerns in many areas. Federated learning is a popular approach for
privacy protection that collects the local gradient information instead of real
data. One way to achieve a strict privacy guarantee is to apply local
differential privacy into federated learning. However, previous works do not
give a practical solution due to three issues. First, the noisy data is close
to its original value with high probability, increasing the risk of information
exposure. Second, a large variance is introduced to the estimated average,
causing poor accuracy. Last, the privacy budget explodes due to the high
dimensionality of weights in deep learning models. In this paper, we proposed a
novel design of local differential privacy mechanism for federated learning to
address the abovementioned issues. It is capable of making the data more
distinct from its original value and introducing lower variance. Moreover, the
proposed mechanism bypasses the curse of dimensionality by splitting and
shuffling model updates. A series of empirical evaluations on three commonly
used datasets, MNIST, Fashion-MNIST and CIFAR-10, demonstrate that our solution
can not only achieve superior deep learning performance but also provide a
strong privacy guarantee at the same time.
