Improving Neural Topic Models using Knowledge Distillation
  Topic models are often used to identify human-interpretable topics to help
make sense of large document collections. We use knowledge distillation to
combine the best attributes of probabilistic topic models and pretrained
transformers. Our modular method can be straightforwardly applied with any
neural topic model to improve topic quality, which we demonstrate using two
models having disparate architectures, obtaining state-of-the-art topic
coherence. We show that our adaptable framework not only improves performance
in the aggregate over all estimated topics, as is commonly reported, but also
in head-to-head comparisons of aligned topics.
