The Spectral Underpinning of word2vec
  word2vec due to Mikolov \textit{et al.} (2013) is a word embedding method
that is widely used in natural language processing. Despite its great success
and frequent use, theoretical justification is still lacking. The main
contribution of our paper is to propose a rigorous analysis of the highly
nonlinear functional of word2vec. Our results suggest that word2vec may be
primarily driven by an underlying spectral method. This insight may open the
door to obtaining provable guarantees for word2vec. We support these findings
by numerical simulations. One fascinating open question is whether the
nonlinear properties of word2vec that are not captured by the spectral method
are beneficial and, if so, by what mechanism.
