Positional Attention-based Frame Identification with BERT: A Deep
  Learning Approach to Target Disambiguation and Semantic Frame Selection
  Semantic parsing is the task of transforming sentences from natural language
into formal representations of predicate-argument structures. Under this
research area, frame-semantic parsing has attracted much interest. This parsing
approach leverages the lexical information defined in FrameNet to associate
marked predicates or targets with semantic frames, thereby assigning semantic
roles to sentence components based on pre-specified frame elements in FrameNet.
In this paper, a deep neural network architecture known as Positional
Attention-based Frame Identification with BERT (PAFIBERT) is presented as a
solution to the frame identification subtask in frame-semantic parsing.
Although the importance of this subtask is well-established, prior research has
yet to find a robust solution that works satisfactorily for both in-domain and
out-of-domain data. This study thus set out to improve frame identification in
light of recent advancements of language modeling and transfer learning in
natural language processing. The proposed method is partially empowered by
BERT, a pre-trained language model that excels at capturing contextual
information in texts. By combining the language representation power of BERT
with a position-based attention mechanism, PAFIBERT is able to attend to
target-specific contexts in sentences for disambiguating targets and
associating them with the most suitable semantic frames. Under various
experimental settings, PAFIBERT outperformed existing solutions by a
significant margin, achieving new state-of-the-art results for both in-domain
and out-of-domain benchmark test sets.
