Scalable Nonparametric Sampling from Multimodal Posteriors with the
  Posterior Bootstrap
  Increasingly complex datasets pose a number of challenges for Bayesian
inference. Conventional posterior sampling based on Markov chain Monte Carlo
can be too computationally intensive, is serial in nature and mixes poorly
between posterior modes. Further, all models are misspecified, which brings
into question the validity of the conventional Bayesian update. We present a
scalable Bayesian nonparametric learning routine that enables posterior
sampling through the optimization of suitably randomized objective functions. A
Dirichlet process prior on the unknown data distribution accounts for model
misspecification, and admits an embarrassingly parallel posterior bootstrap
algorithm that generates independent and exact samples from the nonparametric
posterior distribution. Our method is particularly adept at sampling from
multimodal posterior distributions via a random restart mechanism. We
demonstrate our method on Gaussian mixture model and sparse logistic regression
examples.
