Hiding Among the Clones: A Simple and Nearly Optimal Analysis of Privacy
  Amplification by Shuffling
  Recent work of Erlingsson, Feldman, Mironov, Raghunathan, Talwar, and
Thakurta [EFMRTT19] demonstrates that random shuffling amplifies differential
privacy guarantees of locally randomized data. Such amplification implies
substantially stronger privacy guarantees for systems in which data is
contributed anonymously [BEMMRLRKTS17] and has lead to significant interest in
the shuffle model of privacy [CSUZZ19; EFMRTT19].
  We show that random shuffling of $n$ data records that are input to
$\varepsilon_0$-differentially private local randomizers results in an
$(O((1-e^{-\varepsilon_0})\sqrt{\frac{e^{\varepsilon_0}\log(1/\delta)}{n}}),
\delta)$-differentially private algorithm. This significantly improves over
previous work and achieves the asymptotically optimal dependence in
$\varepsilon_0$. Our result is based on a new approach that is simpler than
previous work and extends to approximate differential privacy with nearly the
same guarantees. Importantly, our work also yields an algorithm for deriving
tighter bounds on the resulting $\varepsilon$ and $\delta$ as well as R\'enyi
differential privacy guarantees. We show numerically that our algorithm gets to
within a small constant factor of the optimal bound. As a direct corollary of
our analysis we derive a simple and nearly optimal algorithm for frequency
estimation in the shuffle model of privacy. We also observe that our result
implies the first asymptotically optimal privacy analysis of noisy stochastic
gradient descent that applies to sampling without replacement.
