Emotion Recognition on large video dataset based on Convolutional
  Feature Extractor and Recurrent Neural Network
  For many years, the emotion recognition task has remained one of the most
interesting and important problems in the field of human-computer interaction.
In this study, we consider the emotion recognition task as a classification as
well as a regression task by processing encoded emotions in different datasets
using deep learning models. Our model combines convolutional neural network
(CNN) with recurrent neural network (RNN) to predict dimensional emotions on
video data. At the first step, CNN extracts feature vectors from video frames.
In the second step, we fed these feature vectors to train RNN for exploiting
the temporal dynamics of video. Furthermore, we analyzed how each neural
network contributes to the system's overall performance. The experiments are
performed on publicly available datasets including the largest modern Aff-Wild2
database. It contains over sixty hours of video data. We discovered the problem
of overfitting of the model on an unbalanced dataset with an illustrative
example using confusion matrices. The problem is solved by downsampling
technique to balance the dataset. By significantly decreasing training data, we
balance the dataset, thereby, the overall performance of the model is improved.
Hence, the study qualitatively describes the abilities of deep learning models
exploring enough amount of data to predict facial emotions. Our proposed method
is implemented using Tensorflow Keras.
