Approximate Inference via Clustering
  In recent years, large-scale Bayesian learning draws a great deal of
attention. However, in big-data era, the amount of data we face is growing much
faster than our ability to deal with it. Fortunately, it is observed that
large-scale datasets usually own rich internal structure and is somewhat
redundant. In this paper, we attempt to simplify the Bayesian posterior via
exploiting this structure. Specifically, we restrict our interest to the
so-called well-clustered datasets and construct an \emph{approximate posterior}
according to the clustering information. Fortunately, the clustering structure
can be efficiently obtained via a particular clustering algorithm. When
constructing the approximate posterior, the data points in the same cluster are
all replaced by the centroid of the cluster. As a result, the posterior can be
significantly simplified. Theoretically, we show that under certain conditions
the approximate posterior we construct is close (measured by KL divergence) to
the exact posterior. Furthermore, thorough experiments are conducted to
validate the fact that the constructed posterior is a good approximation to the
true posterior and much easier to sample from.
