Privacy Amplification Via Bernoulli Sampling
  Balancing privacy and accuracy is a major challenge in designing
differentially private machine learning algorithms. One way to improve this
tradeoff for free is to leverage the noise in common data operations that
already use randomness. Such operations include noisy SGD and data subsampling.
The additional noise in these operations may amplify the privacy guarantee of
the overall algorithm, a phenomenon known as privacy amplification. In this
paper, we analyze the privacy amplification of sampling from a multidimensional
Bernoulli distribution family given the parameter from a private algorithm.
This setup has applications to Bayesian inference and to data compression. We
provide an algorithm to compute the amplification factor, and we establish
upper and lower bounds on this factor.
