LIDSNet: A Lightweight on-device Intent Detection model using Deep
  Siamese Network
  Intent detection is a crucial task in any Natural Language Understanding
(NLU) system and forms the foundation of a task-oriented dialogue system. To
build high-quality real-world conversational solutions for edge devices, there
is a need for deploying intent detection model on device. This necessitates a
light-weight, fast, and accurate model that can perform efficiently in a
resource-constrained environment. To this end, we propose LIDSNet, a novel
lightweight on-device intent detection model, which accurately predicts the
message intent by utilizing a Deep Siamese Network for learning better sentence
representations. We use character-level features to enrich the sentence-level
representations and empirically demonstrate the advantage of transfer learning
by utilizing pre-trained embeddings. Furthermore, to investigate the efficacy
of the modules in our architecture, we conduct an ablation study and arrive at
our optimal model. Experimental results prove that LIDSNet achieves
state-of-the-art competitive accuracy of 98.00% and 95.97% on SNIPS and ATIS
public datasets respectively, with under 0.59M parameters. We further benchmark
LIDSNet against fine-tuned BERTs and show that our model is at least 41x
lighter and 30x faster during inference than MobileBERT on Samsung Galaxy S20
device, justifying its efficiency on resource-constrained edge devices.
