Iterate & Cluster: Iterative Semi-Supervised Action Recognition
  We propose a novel system for active semi-supervised feature-based action
recognition. Given time sequences of features tracked during movements our
system clusters the sequences into actions. Our system is based on
encoder-decoder unsupervised methods shown to perform clustering by
self-organization of their latent representation through the auto-regression
task. These methods were tested on human action recognition benchmarks and
outperformed non-feature based unsupervised methods and achieved comparable
accuracy to skeleton-based supervised methods. However, such methods rely on
K-Nearest Neighbours (KNN) associating sequences to actions, and general
features with no annotated data would correspond to approximate clusters which
could be further enhanced. Our system proposes an iterative semi-supervised
method to address this challenge and to actively learn the association of
clusters and actions. The method utilizes latent space embedding and clustering
of the unsupervised encoder-decoder to guide the selection of sequences to be
annotated in each iteration. Each iteration, the selection aims to enhance
action recognition accuracy while choosing a small number of sequences for
annotation. We test the approach on human skeleton-based action recognition
benchmarks assuming that only annotations chosen by our method are available
and on mouse movements videos recorded in lab experiments. We show that our
system can boost recognition performance with only a small percentage of
annotations. The system can be used as an interactive annotation tool to guide
labeling efforts for 'in the wild' videos of various objects and actions to
reach robust recognition.
