Jointly Learning Word Embeddings and Latent Topics
  Word embedding models such as Skip-gram learn a vector-space representation
for each word, based on the local word collocation patterns that are observed
in a text corpus. Latent topic models, on the other hand, take a more global
view, looking at the word distributions across the corpus to assign a topic to
each word occurrence. These two paradigms are complementary in how they
represent the meaning of word occurrences. While some previous works have
already looked at using word embeddings for improving the quality of latent
topics, and conversely, at using latent topics for improving word embeddings,
such "two-step" methods cannot capture the mutual interaction between the two
paradigms. In this paper, we propose STE, a framework which can learn word
embeddings and latent topics in a unified manner. STE naturally obtains
topic-specific word embeddings, and thus addresses the issue of polysemy. At
the same time, it also learns the term distributions of the topics, and the
topic distributions of the documents. Our experimental results demonstrate that
the STE model can indeed generate useful topic-specific word embeddings and
coherent latent topics in an effective and efficient way.
