Hybrid Attention-Based Transformer Block Model for Distant Supervision
  Relation Extraction
  With an exponential explosive growth of various digital text information, it
is challenging to efficiently obtain specific knowledge from massive
unstructured text information. As one basic task for natural language
processing (NLP), relation extraction aims to extract the semantic relation
between entity pairs based on the given text. To avoid manual labeling of
datasets, distant supervision relation extraction (DSRE) has been widely used,
aiming to utilize knowledge base to automatically annotate datasets.
Unfortunately, this method heavily suffers from wrong labelling due to the
underlying strong assumptions. To address this issue, we propose a new
framework using hybrid attention-based Transformer block with multi-instance
learning to perform the DSRE task. More specifically, the Transformer block is
firstly used as the sentence encoder to capture syntactic information of
sentences, which mainly utilizes multi-head self-attention to extract features
from word level. Then, a more concise sentence-level attention mechanism is
adopted to constitute the bag representation, aiming to incorporate valid
information of each sentence to effectively represent the bag. Experimental
results on the public dataset New York Times (NYT) demonstrate that the
proposed approach can outperform the state-of-the-art algorithms on the
evaluation dataset, which verifies the effectiveness of our model for the DSRE
task.
