Differentially Private ADMM Algorithms for Machine Learning
  In this paper, we study efficient differentially private alternating
direction methods of multipliers (ADMM) via gradient perturbation for many
machine learning problems. For smooth convex loss functions with (non)-smooth
regularization, we propose the first differentially private ADMM (DP-ADMM)
algorithm with performance guarantee of $(\epsilon,\delta)$-differential
privacy ($(\epsilon,\delta)$-DP). From the viewpoint of theoretical analysis,
we use the Gaussian mechanism and the conversion relationship between R\'enyi
Differential Privacy (RDP) and DP to perform a comprehensive privacy analysis
for our algorithm. Then we establish a new criterion to prove the convergence
of the proposed algorithms including DP-ADMM. We also give the utility analysis
of our DP-ADMM. Moreover, we propose an accelerated DP-ADMM (DP-AccADMM) with
the Nesterov's acceleration technique. Finally, we conduct numerical
experiments on many real-world datasets to show the privacy-utility tradeoff of
the two proposed algorithms, and all the comparative analysis shows that
DP-AccADMM converges faster and has a better utility than DP-ADMM, when the
privacy budget $\epsilon$ is larger than a threshold.
