A Financial Service Chatbot based on Deep Bidirectional Transformers
  We develop a chatbot using Deep Bidirectional Transformer models (BERT) to
handle client questions in financial investment customer service. The bot can
recognize 381 intents, and decides when to say "I don't know" and escalates
irrelevant/uncertain questions to human operators. Our main novel contribution
is the discussion about uncertainty measure for BERT, where three different
approaches are systematically compared on real problems. We investigated two
uncertainty metrics, information entropy and variance of dropout sampling in
BERT, followed by mixed-integer programming to optimize decision thresholds.
Another novel contribution is the usage of BERT as a language model in
automatic spelling correction. Inputs with accidental spelling errors can
significantly decrease intent classification performance. The proposed approach
combines probabilities from masked language model and word edit distances to
find the best corrections for misspelled words. The chatbot and the entire
conversational AI system are developed using open-source tools, and deployed
within our company's intranet. The proposed approach can be useful for
industries seeking similar in-house solutions in their specific business
domains. We share all our code and a sample chatbot built on a public dataset
on Github.
