On a Novel Application of Wasserstein-Procrustes for Unsupervised
  Cross-Lingual Learning
  The emergence of unsupervised word embeddings, pre-trained on very large
monolingual text corpora, is at the core of the ongoing neural revolution in
Natural Language Processing (NLP). Initially introduced for English, such
pre-trained word embeddings quickly emerged for a number of other languages.
Subsequently, there have been a number of attempts to align the embedding
spaces across languages, which could enable a number of cross-language NLP
applications. Performing the alignment using unsupervised cross-lingual
learning (UCL) is especially attractive as it requires little data and often
rivals supervised and semi-supervised approaches. Here, we analyze popular
methods for UCL and we find that often their objectives are, intrinsically,
versions of the Wasserstein-Procrustes problem. Hence, we devise an approach to
solve Wasserstein-Procrustes in a direct way, which can be used to refine and
to improve popular UCL methods such as iterative closest point (ICP),
multilingual unsupervised and supervised embeddings (MUSE) and supervised
Procrustes methods. Our evaluation experiments on standard datasets show
sizable improvements over these approaches. We believe that our rethinking of
the Wasserstein-Procrustes problem could enable further research, thus helping
to develop better algorithms for aligning word embeddings across languages. Our
code and instructions to reproduce the experiments are available at
https://github.com/guillemram97/wp-hungarian.
