Improving Word Translation via Two-Stage Contrastive Learning
  Word translation or bilingual lexicon induction (BLI) is a key cross-lingual
task, aiming to bridge the lexical gap between different languages. In this
work, we propose a robust and effective two-stage contrastive learning
framework for the BLI task. At Stage C1, we propose to refine standard
cross-lingual linear maps between static word embeddings (WEs) via a
contrastive learning objective; we also show how to integrate it into the
self-learning procedure for even more refined cross-lingual maps. In Stage C2,
we conduct BLI-oriented contrastive fine-tuning of mBERT, unlocking its word
translation capability. We also show that static WEs induced from the
`C2-tuned' mBERT complement static WEs from Stage C1. Comprehensive experiments
on standard BLI datasets for diverse languages and different experimental
setups demonstrate substantial gains achieved by our framework. While the BLI
method from Stage C1 already yields substantial gains over all state-of-the-art
BLI methods in our comparison, even stronger improvements are met with the full
two-stage framework: e.g., we report gains for 112/112 BLI setups, spanning 28
language pairs.
