On the Differentially Private Nature of Perturbed Gradient Descent
  We consider the problem of empirical risk minimization given a database,
using the gradient descent algorithm. We note that the function to be optimized
may be non-convex, consisting of saddle points which impede the convergence of
the algorithm. A perturbed gradient descent algorithm is typically employed to
escape these saddle points. We show that this algorithm, that perturbs the
gradient, inherently preserves the privacy of the data. We then employ the
differential privacy framework to quantify the privacy hence achieved. We also
analyze the change in privacy with varying parameters such as problem dimension
and the distance between the databases.
