Nonparametric Topic Modeling with Neural Inference
  This work focuses on combining nonparametric topic models with Auto-Encoding
Variational Bayes (AEVB). Specifically, we first propose iTM-VAE, where the
topics are treated as trainable parameters and the document-specific topic
proportions are obtained by a stick-breaking construction. The inference of
iTM-VAE is modeled by neural networks such that it can be computed in a simple
feed-forward manner. We also describe how to introduce a hyper-prior into
iTM-VAE so as to model the uncertainty of the prior parameter. Actually, the
hyper-prior technique is quite general and we show that it can be applied to
other AEVB based models to alleviate the {\it collapse-to-prior} problem
elegantly. Moreover, we also propose HiTM-VAE, where the document-specific
topic distributions are generated in a hierarchical manner. HiTM-VAE is even
more flexible and can generate topic distributions with better variability.
Experimental results on 20News and Reuters RCV1-V2 datasets show that the
proposed models outperform the state-of-the-art baselines significantly. The
advantages of the hyper-prior technique and the hierarchical model construction
are also confirmed by experiments.
