DP-ADMM: ADMM-based Distributed Learning with Differential Privacy
  Alternating Direction Method of Multipliers (ADMM) is a widely used tool for
machine learning in distributed settings, where a machine learning model is
trained over distributed data sources through an interactive process of local
computation and message passing. Such an iterative process could cause privacy
concerns of data owners. The goal of this paper is to provide differential
privacy for ADMM-based distributed machine learning. Prior approaches on
differentially private ADMM exhibit low utility under high privacy guarantee
and often assume the objective functions of the learning problems to be smooth
and strongly convex. To address these concerns, we propose a novel
differentially private ADMM-based distributed learning algorithm called
DP-ADMM, which combines an approximate augmented Lagrangian function with
time-varying Gaussian noise addition in the iterative process to achieve higher
utility for general objective functions under the same differential privacy
guarantee. We also apply the moments accountant method to bound the end-to-end
privacy loss. The theoretical analysis shows that DP-ADMM can be applied to a
wider class of distributed learning problems, is provably convergent, and
offers an explicit utility-privacy tradeoff. To our knowledge, this is the
first paper to provide explicit convergence and utility properties for
differentially private ADMM-based distributed learning algorithms. The
evaluation results demonstrate that our approach can achieve good convergence
and model accuracy under high end-to-end differential privacy guarantee.
