UPB at SemEval-2020 Task 12: Multilingual Offensive Language Detection
  on Social Media by Fine-tuning a Variety of BERT-based Models
  Offensive language detection is one of the most challenging problem in the
natural language processing field, being imposed by the rising presence of this
phenomenon in online social media. This paper describes our Transformer-based
solutions for identifying offensive language on Twitter in five languages
(i.e., English, Arabic, Danish, Greek, and Turkish), which was employed in
Subtask A of the Offenseval 2020 shared task. Several neural architectures
(i.e., BERT, mBERT, Roberta, XLM-Roberta, and ALBERT), pre-trained using both
single-language and multilingual corpora, were fine-tuned and compared using
multiple combinations of datasets. Finally, the highest-scoring models were
used for our submissions in the competition, which ranked our team 21st of 85,
28th of 53, 19th of 39, 16th of 37, and 10th of 46 for English, Arabic, Danish,
Greek, and Turkish, respectively.
