Few Shot Activity Recognition Using Variational Inference
  There has been a remarkable progress in learning a model which could
recognise novel classes with only a few labeled examples in the last few years.
Few-shot learning (FSL) for action recognition is a challenging task of
recognising novel action categories which are represented by few instances in
the training data. We propose a novel variational inference based architectural
framework (HF-AR) for few shot activity recognition. Our framework leverages
volume-preserving Householder Flow to learn a flexible posterior distribution
of the novel classes. This results in better performance as compared to
state-of-the-art few shot approaches for human activity recognition. approach
consists of base model and an adapter model. Our architecture consists of a
base model and an adapter model. The base model is trained on seen classes and
it computes an embedding that represent the spatial and temporal insights
extracted from the input video, e.g. combination of Resnet-152 and LSTM based
encoder-decoder model. The adapter model applies a series of Householder
transformations to compute a flexible posterior distribution that lends higher
accuracy in the few shot approach. Extensive experiments on three well-known
datasets: UCF101, HMDB51 and Something-Something-V2, demonstrate similar or
better performance on 1-shot and 5-shot classification as compared to
state-of-the-art few shot approaches that use only RGB frame sequence as input.
To the best of our knowledge, we are the first to explore variational inference
along with householder transformations to capture the full rank covariance
matrix of posterior distribution, for few shot learning in activity
recognition.
