Topic Modeling of Hierarchical Corpora
  We study the problem of topic modeling in corpora whose documents are
organized in a multi-level hierarchy. We explore a parametric approach to this
problem, assuming that the number of topics is known or can be estimated by
cross-validation. The models we consider can be viewed as special
(finite-dimensional) instances of hierarchical Dirichlet processes (HDPs). For
these models we show that there exists a simple variational approximation for
probabilistic inference. The approximation relies on a previously unexploited
inequality that handles the conditional dependence between Dirichlet latent
variables in adjacent levels of the model's hierarchy. We compare our approach
to existing implementations of nonparametric HDPs. On several benchmarks we
find that our approach is faster than Gibbs sampling and able to learn more
predictive models than existing variational methods. Finally, we demonstrate
the large-scale viability of our approach on two newly available corpora from
researchers in computer security---one with 350,000 documents and over 6,000
internal subcategories, the other with a five-level deep hierarchy.
