TransWiC at SemEval-2021 Task 2: Transformer-based Multilingual and
  Cross-lingual Word-in-Context Disambiguation
  Identifying whether a word carries the same meaning or different meaning in
two contexts is an important research area in natural language processing which
plays a significant role in many applications such as question answering,
document summarisation, information retrieval and information extraction. Most
of the previous work in this area rely on language-specific resources making it
difficult to generalise across languages. Considering this limitation, our
approach to SemEval-2021 Task 2 is based only on pretrained transformer models
and does not use any language-specific processing and resources. Despite that,
our best model achieves 0.90 accuracy for English-English subtask which is very
compatible compared to the best result of the subtask; 0.93 accuracy. Our
approach also achieves satisfactory results in other monolingual and
cross-lingual language pairs as well.
