AdaCliP: Adaptive Clipping for Private SGD
  Privacy preserving machine learning algorithms are crucial for learning
models over user data to protect sensitive information. Motivated by this,
differentially private stochastic gradient descent (SGD) algorithms for
training machine learning models have been proposed. At each step, these
algorithms modify the gradients and add noise proportional to the sensitivity
of the modified gradients. Under this framework, we propose AdaCliP, a
theoretically motivated differentially private SGD algorithm that provably adds
less noise compared to the previous methods, by using coordinate-wise adaptive
clipping of the gradient. We empirically demonstrate that AdaCliP reduces the
amount of added noise and produces models with better accuracy.
