On SkipGram Word Embedding Models with Negative Sampling: Unified
  Framework and Impact of Noise Distributions
  SkipGram word embedding models with negative sampling, or SGN in short, is an
elegant family of word embedding models. In this paper, we formulate a
framework for word embedding, referred to as Word-Context Classification (WCC),
that generalizes SGN to a wide family of models. The framework, utilizing some
"noise examples", is justified through a theoretical analysis. The impact of
noise distribution on the learning of the WCC embedding models is studied
experimentally, suggesting that the best noise distribution is in fact the data
distribution, in terms of both the embedding performance and the speed of
convergence during training. Along our way, we discover several novel embedding
models that outperform the existing WCC models.
