Ensemble Slice Sampling: Parallel, black-box and gradient-free inference
  for correlated & multimodal distributions
  Slice Sampling has emerged as a powerful Markov Chain Monte Carlo algorithm
that adapts to the characteristics of the target distribution with minimal
hand-tuning. However, Slice Sampling's performance is highly sensitive to the
user-specified initial length scale hyperparameter and the method generally
struggles with poorly scaled or strongly correlated distributions. This paper
introduces Ensemble Slice Sampling (ESS), a new class of algorithms that
bypasses such difficulties by adaptively tuning the initial length scale and
utilising an ensemble of parallel walkers in order to efficiently handle strong
correlations between parameters. These affine-invariant algorithms are trivial
to construct, require no hand-tuning, and can easily be implemented in parallel
computing environments. Empirical tests show that Ensemble Slice Sampling can
improve efficiency by more than an order of magnitude compared to conventional
MCMC methods on a broad range of highly correlated target distributions. In
cases of strongly multimodal target distributions, Ensemble Slice Sampling can
sample efficiently even in high dimensions. We argue that the parallel,
black-box and gradient-free nature of the method renders it ideal for use in
scientific fields such as physics, astrophysics and cosmology which are
dominated by a wide variety of computationally expensive and non-differentiable
models.
