Generative Adversarial Networks for Annotated Data Augmentation in Data
  Sparse NLU
  Data sparsity is one of the key challenges associated with model development
in Natural Language Understanding (NLU) for conversational agents. The
challenge is made more complex by the demand for high quality annotated
utterances commonly required for supervised learning, usually resulting in
weeks of manual labor and high cost. In this paper, we present our results on
boosting NLU model performance through training data augmentation using a
sequential generative adversarial network (GAN). We explore data generation in
the context of two tasks, the bootstrapping of a new language and the handling
of low resource features. For both tasks we explore three sequential GAN
architectures, one with a token-level reward function, another with our own
implementation of a token-level Monte Carlo rollout reward, and a third with
sentence-level reward. We evaluate the performance of these feedback models
across several sampling methodologies and compare our results to upsampling the
original data to the same scale. We further improve the GAN model performance
through the transfer learning of the pretrained embeddings. Our experiments
reveal synthetic data generated using the sequential generative adversarial
network provides significant performance boosts across multiple metrics and can
be a major benefit to the NLU tasks.
