Faster Differentially Private Samplers via R\'enyi Divergence Analysis
  of Discretized Langevin MCMC
  Various differentially private algorithms instantiate the exponential
mechanism, and require sampling from the distribution $\exp(-f)$ for a suitable
function $f$. When the domain of the distribution is high-dimensional, this
sampling can be computationally challenging. Using heuristic sampling schemes
such as Gibbs sampling does not necessarily lead to provable privacy. When $f$
is convex, techniques from log-concave sampling lead to polynomial-time
algorithms, albeit with large polynomials. Langevin dynamics-based algorithms
offer much faster alternatives under some distance measures such as statistical
distance. In this work, we establish rapid convergence for these algorithms
under distance measures more suitable for differential privacy. For smooth,
strongly-convex $f$, we give the first results proving convergence in R\'enyi
divergence. This gives us fast differentially private algorithms for such $f$.
Our techniques and simple and generic and apply also to underdamped Langevin
dynamics.
