Temporal Stochastic Softmax for 3D CNNs: An Application in Facial
  Expression Recognition
  Training deep learning models for accurate spatiotemporal recognition of
facial expressions in videos requires significant computational resources. For
practical reasons, 3D Convolutional Neural Networks (3D CNNs) are usually
trained with relatively short clips randomly extracted from videos. However,
such uniform sampling is generally sub-optimal because equal importance is
assigned to each temporal clip. In this paper, we present a strategy for
efficient video-based training of 3D CNNs. It relies on softmax temporal
pooling and a weighted sampling mechanism to select the most relevant training
clips. The proposed softmax strategy provides several advantages: a reduced
computational complexity due to efficient clip sampling, and an improved
accuracy since temporal weighting focuses on more relevant clips during both
training and inference. Experimental results obtained with the proposed method
on several facial expression recognition benchmarks show the benefits of
focusing on more informative clips in training videos. In particular, our
approach improves performance and computational cost by reducing the impact of
inaccurate trimming and coarse annotation of videos, and heterogeneous
distribution of visual information across time.
