A Post-processing Method for Detecting Unknown Intent of Dialogue System
  via Pre-trained Deep Neural Network Classifier
  With the maturity and popularity of dialogue systems, detecting user's
unknown intent in dialogue systems has become an important task. It is also one
of the most challenging tasks since we can hardly get examples, prior knowledge
or the exact numbers of unknown intents. In this paper, we propose SofterMax
and deep novelty detection (SMDN), a simple yet effective post-processing
method for detecting unknown intent in dialogue systems based on pre-trained
deep neural network classifiers. Our method can be flexibly applied on top of
any classifiers trained in deep neural networks without changing the model
architecture. We calibrate the confidence of the softmax outputs to compute the
calibrated confidence score (i.e., SofterMax) and use it to calculate the
decision boundary for unknown intent detection. Furthermore, we feed the
feature representations learned by the deep neural networks into traditional
novelty detection algorithm to detect unknown intents from different
perspectives. Finally, we combine the methods above to perform the joint
prediction. Our method classifies examples that differ from known intents as
unknown and does not require any examples or prior knowledge of it. We have
conducted extensive experiments on three benchmark dialogue datasets. The
results show that our method can yield significant improvements compared with
the state-of-the-art baselines
