Multi-modal fusion with gating using audio, lexical and disfluency
  features for Alzheimer's Dementia recognition from spontaneous speech
  This paper is a submission to the Alzheimer's Dementia Recognition through
Spontaneous Speech (ADReSS) challenge, which aims to develop methods that can
assist in the automated prediction of severity of Alzheimer's Disease from
speech data. We focus on acoustic and natural language features for cognitive
impairment detection in spontaneous speech in the context of Alzheimer's
Disease Diagnosis and the mini-mental state examination (MMSE) score
prediction. We proposed a model that obtains unimodal decisions from different
LSTMs, one for each modality of text and audio, and then combines them using a
gating mechanism for the final prediction. We focused on sequential modelling
of text and audio and investigated whether the disfluencies present in
individuals' speech relate to the extent of their cognitive impairment. Our
results show that the proposed classification and regression schemes obtain
very promising results on both development and test sets. This suggests
Alzheimer's Disease can be detected successfully with sequence modeling of the
speech data of medical sessions.
