Mixed Membership Word Embeddings for Computational Social Science
  Word embeddings improve the performance of NLP systems by revealing the
hidden structural relationships between words. Despite their success in many
applications, word embeddings have seen very little use in computational social
science NLP tasks, presumably due to their reliance on big data, and to a lack
of interpretability. I propose a probabilistic model-based word embedding
method which can recover interpretable embeddings, without big data. The key
insight is to leverage mixed membership modeling, in which global
representations are shared, but individual entities (i.e. dictionary words) are
free to use these representations to uniquely differing degrees. I show how to
train the model using a combination of state-of-the-art training techniques for
word embeddings and topic models. The experimental results show an improvement
in predictive language modeling of up to 63% in MRR over the skip-gram, and
demonstrate that the representations are beneficial for supervised learning. I
illustrate the interpretability of the models with computational social science
case studies on State of the Union addresses and NIPS articles.
