Improved Customer Transaction Classification using Semi-Supervised
  Knowledge Distillation
  In pickup and delivery services, transaction classification based on customer
provided free text is a challenging problem. It involves the association of a
wide variety of customer inputs to a fixed set of categories while adapting to
the various customer writing styles. This categorization is important for the
business: it helps understand the market needs and trends, and also assist in
building a personalized experience for different segments of the customers.
Hence, it is vital to capture these category information trends at scale, with
high precision and recall. In this paper, we focus on a specific use-case where
a single category drives each transaction. We propose a cost-effective
transaction classification approach based on semi-supervision and knowledge
distillation frameworks. The approach identifies the category of a transaction
using free text input given by the customer. We use weak labelling and notice
that the performance gains are similar to that of using human-annotated
samples. On a large internal dataset and on 20Newsgroup dataset, we see that
RoBERTa performs the best for the categorization tasks. Further, using an
ALBERT model (it has 33x fewer parameters vis-a-vis parameters of RoBERTa),
with RoBERTa as the Teacher, we see a performance similar to that of RoBERTa
and better performance over unadapted ALBERT. This framework, with ALBERT as a
student and RoBERTa as teacher, is further referred to as R-ALBERT in this
paper. The model is in production and is used by business to understand
changing trends and take appropriate decisions.
