Prior Knowledge Driven Label Embedding for Slot Filling in Natural
  Language Understanding
  Traditional slot filling in natural language understanding (NLU) predicts a
one-hot vector for each word. This form of label representation lacks semantic
correlation modelling, which leads to severe data sparsity problem, especially
when adapting an NLU model to a new domain. To address this issue, a novel
label embedding based slot filling framework is proposed in this paper. Here,
distributed label embedding is constructed for each slot using prior knowledge.
Three encoding methods are investigated to incorporate different kinds of prior
knowledge about slots: atomic concepts, slot descriptions, and slot exemplars.
The proposed label embeddings tend to share text patterns and reuses data with
different slot labels. This makes it useful for adaptive NLU with limited data.
Also, since label embedding is independent of NLU model, it is compatible with
almost all deep learning based slot filling models. The proposed approaches are
evaluated on three datasets. Experiments on single domain and domain adaptation
tasks show that label embedding achieves significant performance improvement
over traditional one-hot label representation as well as advanced zero-shot
approaches.
