Measuring Sample Quality with Stein's Method
  To improve the efficiency of Monte Carlo estimation, practitioners are
turning to biased Markov chain Monte Carlo procedures that trade off asymptotic
exactness for computational speed. The reasoning is sound: a reduction in
variance due to more rapid sampling can outweigh the bias introduced. However,
the inexactness creates new challenges for sampler and parameter selection,
since standard measures of sample quality like effective sample size do not
account for asymptotic bias. To address these challenges, we introduce a new
computable quality measure based on Stein's method that quantifies the maximum
discrepancy between sample and target expectations over a large class of test
functions. We use our tool to compare exact, biased, and deterministic sample
sequences and illustrate applications to hyperparameter selection, convergence
rate assessment, and quantifying bias-variance tradeoffs in posterior
inference.
