RGB Video Based Tennis Action Recognition Using a Deep Historical Long
  Short-Term Memory
  Action recognition has attracted increasing attention from RGB input in
computer vision partially due to potential applications on somatic simulation
and statistics of sport such as virtual tennis game and tennis techniques and
tactics analysis by video. Recently, deep learning based methods have achieved
promising performance for action recognition. In this paper, we propose
weighted Long Short-Term Memory adopted with convolutional neural network
representations for three dimensional tennis shots recognition. First, the
local two-dimensional convolutional neural network spatial representations are
extracted from each video frame individually using a pre-trained Inception
network. Then, a weighted Long Short-Term Memory decoder is introduced to take
the output state at time t and the historical embedding feature at time t-1 to
generate feature vector using a score weighting scheme. Finally, we use the
adopted CNN and weighted LSTM to map the original visual features into a vector
space to generate the spatial-temporal semantical description of visual
sequences and classify the action video content. Experiments on the benchmark
demonstrate that our method using only simple raw RGB video can achieve better
performance than the state-of-the-art baselines for tennis shot recognition.
