Attentive Cross-modal Connections for Deep Multimodal Wearable-based
  Emotion Recognition
  Classification of human emotions can play an essential role in the design and
improvement of human-machine systems. While individual biological signals such
as Electrocardiogram (ECG) and Electrodermal Activity (EDA) have been widely
used for emotion recognition with machine learning methods, multimodal
approaches generally fuse extracted features or final classification/regression
results to boost performance. To enhance multimodal learning, we present a
novel attentive cross-modal connection to share information between
convolutional neural networks responsible for learning individual modalities.
Specifically, these connections improve emotion classification by sharing
intermediate representations among EDA and ECG and apply attention weights to
the shared information, thus learning more effective multimodal embeddings. We
perform experiments on the WESAD dataset to identify the best configuration of
the proposed method for emotion classification. Our experiments show that the
proposed approach is capable of learning strong multimodal representations and
outperforms a number of baselines methods.
