Dependency Parsing for Spoken Dialog Systems
  Dependency parsing of conversational input can play an important role in
language understanding for dialog systems by identifying the relationships
between entities extracted from user utterances. Additionally, effective
dependency parsing can elucidate differences in language structure and usage
for discourse analysis of human-human versus human-machine dialogs. However,
models trained on datasets based on news articles and web data do not perform
well on spoken human-machine dialog, and currently available annotation schemes
do not adapt well to dialog data. Therefore, we propose the Spoken Conversation
Universal Dependencies (SCUD) annotation scheme that extends the Universal
Dependencies (UD) (Nivre et al., 2016) guidelines to spoken human-machine
dialogs. We also provide ConvBank, a conversation dataset between humans and an
open-domain conversational dialog system with SCUD annotation. Finally, to
demonstrate the utility of the dataset, we train a dependency parser on the
ConvBank dataset. We demonstrate that by pre-training a dependency parser on a
set of larger public datasets and fine-tuning on ConvBank data, we achieved the
best result, 85.05% unlabeled and 77.82% labeled attachment accuracy.
