Word2Vec: Optimal Hyper-Parameters and Their Impact on NLP Downstream
  Tasks
  Word2Vec is a prominent model for natural language processing (NLP) tasks.
Similar inspiration is found in distributed embeddings for new state-of-the-art
(SotA) deep neural networks. However, wrong combination of hyper-parameters can
produce poor quality vectors. The objective of this work is to empirically show
optimal combination of hyper-parameters exists and evaluate various
combinations. We compare them with the released, pre-trained original word2vec
model. Both intrinsic and extrinsic (downstream) evaluations, including named
entity recognition (NER) and sentiment analysis (SA) were carried out. The
downstream tasks reveal that the best model is usually task-specific, high
analogy scores don't necessarily correlate positively with F1 scores and the
same applies to focus on data alone. Increasing vector dimension size after a
point leads to poor quality or performance. If ethical considerations to save
time, energy and the environment are made, then reasonably smaller corpora may
do just as well or even better in some cases. Besides, using a small corpus, we
obtain better human-assigned WordSim scores, corresponding Spearman correlation
and better downstream performances (with significance tests) compared to the
original model, trained on 100 billion-word corpus.
