Towards a Universal NLG for Dialogue Systems and Simulators with Future
  Bridging
  In a dialogue system pipeline, a natural language generation (NLG) unit
converts the dialogue direction and content to a corresponding natural language
realization. A recent trend for dialogue systems is to first pre-train on large
datasets and then fine-tune in a supervised manner using datasets annotated
with application-specific features. Though novel behaviours can be learned from
custom annotation, the required effort severely bounds the quantity of the
training set, and the application-specific nature limits the reuse. In light of
the recent success of data-driven approaches, we propose the novel future
bridging NLG (FBNLG) concept for dialogue systems and simulators. The critical
step is for an FBNLG to accept a future user or system utterance to bridge the
present context towards. Future bridging enables self supervised training over
annotation-free datasets, decoupled the training of NLG from the rest of the
system. An FBNLG, pre-trained with massive datasets, is expected to apply in
classical or new dialogue scenarios with minimal adaptation effort. We evaluate
a prototype FBNLG to show that future bridging can be a viable approach to a
universal few-shot NLG for task-oriented and chit-chat dialogues.
