A Flexible Clustering Pipeline for Mining Text Intentions
  Mining the latent intentions from large volumes of natural language inputs is
a key step to help data analysts design and refine Intelligent Virtual
Assistants (IVAs) for customer service and sales support. We created a flexible
and scalable clustering pipeline within the Verint Intent Manager (VIM) that
integrates the fine-tuning of language models, a high performing k-NN library
and community detection techniques to help analysts quickly surface and
organize relevant user intentions from conversational texts. The fine-tuning
step is necessary because pre-trained language models cannot encode texts to
efficiently surface particular clustering structures when the target texts are
from an unseen domain or the clustering task is not topic detection. We
describe the pipeline and demonstrate its performance using BERT on three
real-world text mining tasks. As deployed in the VIM application, this
clustering pipeline produces high quality results, improving the performance of
data analysts and reducing the time it takes to surface intentions from
customer service data, thereby reducing the time it takes to build and deploy
IVAs in new domains.
