WOVe: Incorporating Word Order in GloVe Word Embeddings
  Word vector representations open up new opportunities to extract useful
information from unstructured text. Defining a word as a vector made it easy
for the machine learning algorithms to understand a text and extract
information from. Word vector representations have been used in many
applications such word synonyms, word analogy, syntactic parsing, and many
others. GloVe, based on word contexts and matrix vectorization, is an
ef-fective vector-learning algorithm. It improves on previous vector-learning
algorithms. However, the GloVe model fails to explicitly consider the order in
which words appear within their contexts. In this paper, multiple methods of
incorporating word order in GloVe word embeddings are proposed. Experimental
results show that our Word Order Vector (WOVe) word embeddings approach
outperforms unmodified GloVe on the natural lan-guage tasks of analogy
completion and word similarity. WOVe with direct concatenation slightly
outperformed GloVe on the word similarity task, increasing average rank by 2%.
However, it greatly improved on the GloVe baseline on a word analogy task,
achieving an average 36.34% improvement in accuracy.
