Constructing a Highlight Classifier with an Attention Based LSTM Neural
  Network
  Data is being produced in larger quantities than ever before in human
history. It's only natural to expect a rise in demand for technology that aids
humans in sifting through and analyzing this inexhaustible supply of
information. This need exists in the market research industry, where large
amounts of consumer research data is collected through video recordings. At
present, the standard method for analyzing video data is human labor. Market
researchers manually review the vast majority of consumer research video in
order to identify relevant portions - highlights. The industry state of the art
turnaround ratio is 2.2 - for every hour of video content 2.2 hours of manpower
are required. In this study we present a novel approach for NLP-based highlight
identification and extraction based on a supervised learning model that aides
market researchers in sifting through their data. Our approach hinges on a
manually curated user-generated highlight clips constructed from long and
short-form video data. The problem is best suited for an NLP approach due to
the availability of video transcription. We evaluate multiple classes of
models, from gradient boosting to recurrent neural networks, comparing their
performance in extraction and identification of highlights. The best performing
models are then evaluated using four sampling methods designed to analyze
documents much larger than the maximum input length of the classifiers. We
report very high performances for the standalone classifiers, ROC AUC scores in
the range 0.93-0.94, but observe a significant drop in effectiveness when
evaluated on large documents. Based on our results we suggest combinations of
models/sampling algorithms for various use cases.
