Exact slice sampler for Hierarchical Dirichlet Processes
  We propose an exact slice sampler for Hierarchical Dirichlet process (HDP)
and its associated mixture models (Teh et al., 2006). Although there are
existing MCMC algorithms for sampling from the HDP, a slice sampler has been
missing from the literature. Slice sampling is well-known for its desirable
properties including its fast mixing and its natural potential for
parallelization. On the other hand, the hierarchical nature of HDPs poses
challenges to adopting a full-fledged slice sampler that automatically
truncates all the infinite measures involved without ad-hoc modifications. In
this work, we adopt the powerful idea of Bayesian variable augmentation to
address this challenge. By introducing new latent variables, we obtain a full
factorization of the joint distribution that is suitable for slice sampling.
Our algorithm has several appealing features such as (1) fast mixing; (2)
remaining exact while allowing natural truncation of the underlying
infinite-dimensional measures, as in (Kalli et al., 2011), resulting in updates
of only a finite number of necessary atoms and weights in each iteration; and
(3) being naturally suited to parallel implementations. The underlying
principle for joint factorization of the full likelihood is simple and can be
applied to many other settings, such as designing sampling algorithms for
general dependent Dirichlet process (DDP) models.
