Not All Neural Embeddings are Born Equal
  Neural language models learn word representations that capture rich linguistic and conceptual information. Here we investigate the embeddings learned by neural machine translation models. We show that translation-based embeddings outperform those learned by cutting-edge monolingual models at single-language tasks requiring knowledge of conceptual similarity and/or syntactic role. The findings suggest that, while monolingual models learn information about how concepts are related, neural-translation models better capture their true ontological status. 
Felix Hill, KyungHyun Cho, Sebastien Jean, Coline Devin and Yoshua   Bengio
cs.CL