Automated Generation of Multilingual Clusters for the Evaluation of   Distributed Representations
  We propose a language-agnostic way of automatically generating sets of semantically similar clusters of entities along with sets of "outlier" elements, which may then be used to perform an intrinsic evaluation of word embeddings in the outlier detection task. We used our methodology to create a gold-standard dataset, which we call WikiSem500, and evaluated multiple state-of-the-art embeddings. The results show a correlation between performance on this dataset and performance on sentiment analysis. 
Philip Blair, Yuval Merhav, and Joel Barry
cs.CL cs.LG