VGCN-BERT: Augmenting BERT with Graph Embedding for Text Classification
  Much progress has been made recently on text classification with methods
based on neural networks. In particular, models using attention mechanism such
as BERT have shown to have the capability of capturing the contextual
information within a sentence or document. However, their ability of capturing
the global information about the vocabulary of a language is more limited. This
latter is the strength of Graph Convolutional Networks (GCN). In this paper, we
propose VGCN-BERT model which combines the capability of BERT with a Vocabulary
Graph Convolutional Network (VGCN). Local information and global information
interact through different layers of BERT, allowing them to influence mutually
and to build together a final representation for classification. In our
experiments on several text classification datasets, our approach outperforms
BERT and GCN alone, and achieve higher effectiveness than that reported in
previous studies.
