End-to-end Learning of LDA by Mirror-Descent Back Propagation over a
  Deep Architecture
  We develop a fully discriminative learning approach for supervised Latent
Dirichlet Allocation (LDA) model using Back Propagation (i.e., BP-sLDA), which
maximizes the posterior probability of the prediction variable given the input
document. Different from traditional variational learning or Gibbs sampling
approaches, the proposed learning method applies (i) the mirror descent
algorithm for maximum a posterior inference and (ii) back propagation over a
deep architecture together with stochastic gradient/mirror descent for model
parameter estimation, leading to scalable and end-to-end discriminative
learning of the model. As a byproduct, we also apply this technique to develop
a new learning method for the traditional unsupervised LDA model (i.e.,
BP-LDA). Experimental results on three real-world regression and classification
tasks show that the proposed methods significantly outperform the previous
supervised topic models, neural networks, and is on par with deep neural
networks.
