Differentially Private ADMM for Convex Distributed Learning: Improved
  Accuracy via Multi-Step Approximation
  Alternating Direction Method of Multipliers (ADMM) is a popular algorithm for
distributed learning, where a network of nodes collaboratively solve a
regularized empirical risk minimization by iterative local computation
associated with distributed data and iterate exchanges. When the training data
is sensitive, the exchanged iterates will cause serious privacy concern. In
this paper, we aim to propose a new differentially private distributed ADMM
algorithm with improved accuracy for a wide range of convex learning problems.
In our proposed algorithm, we adopt the approximation of the objective function
in the local computation to introduce calibrated noise into iterate updates
robustly, and allow multiple primal variable updates per node in each
iteration. Our theoretical results demonstrate that our approach can obtain
higher utility by such multiple approximate updates, and achieve the error
bounds asymptotic to the state-of-art ones for differentially private empirical
risk minimization.
