Stability Enhanced Privacy and Applications in Private Stochastic
  Gradient Descent
  Private machine learning involves addition of noise while training, resulting
in lower accuracy. Intuitively, greater stability can imply greater privacy and
improve this privacy-utility tradeoff. We study this role of stability in
private empirical risk minimization, where differential privacy is achieved by
output perturbation, and establish a corresponding theoretical result showing
that for strongly-convex loss functions, an algorithm with uniform stability of
$\beta$ implies a bound of $O(\sqrt{\beta})$ on the scale of noise required for
differential privacy.
  The result applies to both explicit regularization and to implicitly
stabilized ERM, such as adaptations of Stochastic Gradient Descent that are
known to be stable. Thus, it generalizes recent results that improve privacy
through modifications to SGD, and establishes stability as the unifying
perspective. It implies new privacy guarantees for optimizations with uniform
stability guarantees, where a corresponding differential privacy guarantee was
previously not known. Experimental results validate the utility of stability
enhanced privacy in several problems, including application of elastic nets and
feature selection.
