Distributed Differentially Private Computation of Functions with
  Correlated Noise
  Many applications of machine learning, such as human health research, involve
processing private or sensitive information. Privacy concerns may impose
significant hurdles to collaboration in scenarios where there are multiple
sites holding data and the goal is to estimate properties jointly across all
datasets. Differentially private decentralized algorithms can provide strong
privacy guarantees. However, the accuracy of the joint estimates may be poor
when the datasets at each site are small. This paper proposes a new framework,
Correlation Assisted Private Estimation (CAPE), for designing
privacy-preserving decentralized algorithms with better accuracy guarantees in
an honest-but-curious model. CAPE can be used in conjunction with the
functional mechanism for statistical and machine learning optimization
problems. A tighter characterization of the functional mechanism is provided
that allows CAPE to achieve the same performance as a centralized algorithm in
the decentralized setting using all datasets. Empirical results on regression
and neural network problems for both synthetic and real datasets show that
differentially private methods can be competitive with non-private algorithms
in many scenarios of interest.
