Augmenting semantic lexicons using word embeddings and transfer learning
  Sentiment-aware intelligent systems are essential to a wide array of
applications. These systems are driven by language models which broadly fall
into two paradigms: Lexicon-based and contextual. Although recent contextual
models are increasingly dominant, we still see demand for lexicon-based models
because of their interpretability and ease of use. For example, lexicon-based
models allow researchers to readily determine which words and phrases
contribute most to a change in measured sentiment. A challenge for any
lexicon-based approach is that the lexicon needs to be routinely expanded with
new words and expressions. Here, we propose two models for automatic lexicon
expansion. Our first model establishes a baseline employing a simple and
shallow neural network initialized with pre-trained word embeddings using a
non-contextual approach. Our second model improves upon our baseline, featuring
a deep Transformer-based network that brings to bear word definitions to
estimate their lexical polarity. Our evaluation shows that both models are able
to score new words with a similar accuracy to reviewers from Amazon Mechanical
Turk, but at a fraction of the cost.
