Residual Belief Propagation for Topic Modeling
  Fast convergence speed is a desired property for training latent Dirichlet
allocation (LDA), especially in online and parallel topic modeling for massive
data sets. This paper presents a novel residual belief propagation (RBP)
algorithm to accelerate the convergence speed for training LDA. The proposed
RBP uses an informed scheduling scheme for asynchronous message passing, which
passes fast-convergent messages with a higher priority to influence those
slow-convergent messages at each learning iteration. Extensive empirical
studies confirm that RBP significantly reduces the training time until
convergence while achieves a much lower predictive perplexity than other
state-of-the-art training algorithms for LDA, including variational Bayes (VB),
collapsed Gibbs sampling (GS), loopy belief propagation (BP), and residual VB
(RVB).
