Priberam Labs at the NTCIR-15 SHINRA2020-ML: Classification Task
  Wikipedia is an online encyclopedia available in 285 languages. It composes
an extremely relevant Knowledge Base (KB), which could be leveraged by
automatic systems for several purposes. However, the structure and organisation
of such information are not prone to automatic parsing and understanding and it
is, therefore, necessary to structure this knowledge. The goal of the current
SHINRA2020-ML task is to leverage Wikipedia pages in order to categorise their
corresponding entities across 268 hierarchical categories, belonging to the
Extended Named Entity (ENE) ontology. In this work, we propose three distinct
models based on the contextualised embeddings yielded by Multilingual BERT. We
explore the performances of a linear layer with and without explicit usage of
the ontology's hierarchy, and a Gated Recurrent Units (GRU) layer. We also test
several pooling strategies to leverage BERT's embeddings and selection criteria
based on the labels' scores. We were able to achieve good performance across a
large variety of languages, including those not seen during the fine-tuning
process (zero-shot languages).
