Unbiased Bayes for Big Data: Paths of Partial Posteriors
  A key quantity of interest in Bayesian inference are expectations of
functions with respect to a posterior distribution. Markov Chain Monte Carlo is
a fundamental tool to consistently compute these expectations via averaging
samples drawn from an approximate posterior. However, its feasibility is being
challenged in the era of so called Big Data as all data needs to be processed
in every iteration. Realising that such simulation is an unnecessarily hard
problem if the goal is estimation, we construct a computationally scalable
methodology that allows unbiased estimation of the required expectations --
without explicit simulation from the full posterior. The scheme's variance is
finite by construction and straightforward to control, leading to algorithms
that are provably unbiased and naturally arrive at a desired error tolerance.
This is achieved at an average computational complexity that is sub-linear in
the size of the dataset and its free parameters are easy to tune. We
demonstrate the utility and generality of the methodology on a range of common
statistical models applied to large-scale benchmark and real-world datasets.
