A Proximal Algorithm for Sampling
  We consider sampling problems with possibly non-smooth potentials (negative
log-densities). In particular, we study two specific settings of sampling where
the convex potential is either semi-smooth or in composite form as the sum of a
smooth component and a semi-smooth component. To overcome the challenges caused
by the non-smoothness, we propose a Markov chain Monte Carlo algorithm that
resembles proximal methods in optimization for these sampling tasks. The key
component of our method is a sampling scheme for a quadratically regularized
target potential. This scheme relies on rejection sampling with a carefully
designed Gaussian proposal whose center is an approximate minimizer of the
regularized potential. We develop a novel technique (a modified Gaussian
integral) to bound the complexity of this rejection sampling scheme in spite of
the non-smoothness in the potentials. We then combine this scheme with the
alternating sampling framework (ASF), which uses Gibbs sampling on an augmented
distribution, to accomplish the two settings of sampling tasks we consider.
Furthermore, by combining the complexity bound of the rejection sampling we
develop and the remarkable convergence properties of ASF discovered recently,
we are able to establish several non-asymptotic complexity bounds for our
algorithm, in terms of the total number of queries of subgradient of the target
potential. Our algorithm achieves state-of-the-art complexity bounds compared
with all existing methods in the same settings.
