Video Summarization with Long Short-term Memory
  We propose a novel supervised learning technique for summarizing videos by
automatically selecting keyframes or key subshots. Casting the problem as a
structured prediction problem on sequential data, our main idea is to use Long
Short-Term Memory (LSTM), a special type of recurrent neural networks to model
the variable-range dependencies entailed in the task of video summarization.
Our learning models attain the state-of-the-art results on two benchmark video
datasets. Detailed analysis justifies the design of the models. In particular,
we show that it is crucial to take into consideration the sequential structures
in videos and model them. Besides advances in modeling techniques, we introduce
techniques to address the need of a large number of annotated data for training
complex learning models. There, our main idea is to exploit the existence of
auxiliary annotated video datasets, albeit heterogeneous in visual styles and
contents. Specifically, we show domain adaptation techniques can improve
summarization by reducing the discrepancies in statistical properties across
those datasets.
