Sense representations for Portuguese: experiments with sense embeddings
  and deep neural language models
  Sense representations have gone beyond word representations like Word2Vec,
GloVe and FastText and achieved innovative performance on a wide range of
natural language processing tasks. Although very useful in many applications,
the traditional approaches for generating word embeddings have a strict
drawback: they produce a single vector representation for a given word ignoring
the fact that ambiguous words can assume different meanings. In this paper, we
explore unsupervised sense representations which, different from traditional
word embeddings, are able to induce different senses of a word by analyzing its
contextual semantics in a text. The unsupervised sense representations
investigated in this paper are: sense embeddings and deep neural language
models. We present the first experiments carried out for generating sense
embeddings for Portuguese. Our experiments show that the sense embedding model
(Sense2vec) outperformed traditional word embeddings in syntactic and semantic
analogies task, proving that the language resource generated here can improve
the performance of NLP tasks in Portuguese. We also evaluated the performance
of pre-trained deep neural language models (ELMo and BERT) in two transfer
learning approaches: feature based and fine-tuning, in the semantic textual
similarity task. Our experiments indicate that the fine tuned Multilingual and
Portuguese BERT language models were able to achieve better accuracy than the
ELMo model and baselines.
