Word2Vec is a special case of Kernel Correspondence Analysis and Kernels
  for Natural Language Processing
  We show that correspondence analysis (CA) is equivalent to defining a Gini
index with appropriately scaled one-hot encoding. Using this relation, we
introduce a nonlinear kernel extension to CA. This extended CA gives a known
analysis for natural language via specialized kernels that use an appropriate
contingency table. We propose a semi-supervised CA, which is a special case of
the kernel extension to CA. Because CA requires excessive memory if applied to
numerous categories, CA has not been used for natural language processing. We
address this problem by introducing delayed evaluation to randomized singular
value decomposition. The memory-efficient CA is then applied to a word-vector
representation task. We propose a tail-cut kernel, which is an extension to the
skip-gram within the kernel extension to CA. Our tail-cut kernel outperforms
existing word-vector representation methods.
