On the Theory and Practice of Privacy-Preserving Bayesian Data Analysis
  Bayesian inference has great promise for the privacy-preserving analysis of
sensitive data, as posterior sampling automatically preserves differential
privacy, an algorithmic notion of data privacy, under certain conditions
(Dimitrakakis et al., 2014; Wang et al., 2015). While this one posterior sample
(OPS) approach elegantly provides privacy "for free," it is data inefficient in
the sense of asymptotic relative efficiency (ARE). We show that a simple
alternative based on the Laplace mechanism, the workhorse of differential
privacy, is as asymptotically efficient as non-private posterior inference,
under general assumptions. This technique also has practical advantages
including efficient use of the privacy budget for MCMC. We demonstrate the
practicality of our approach on a time-series analysis of sensitive military
records from the Afghanistan and Iraq wars disclosed by the Wikileaks
organization.
