Speaker Adaptation Using Spectro-Temporal Deep Features for Dysarthric
  and Elderly Speech Recognition
  Despite the rapid progress of automatic speech recognition (ASR) technologies
targeting normal speech in recent decades, accurate recognition of dysarthric
and elderly speech remains highly challenging tasks to date. Sources of
heterogeneity commonly found in normal speech including accent or gender, when
further compounded with the variability over age and speech pathology severity
level, create large diversity among speakers. To this end, speaker adaptation
techniques play a key role in personalization of ASR systems for such users.
Motivated by the spectro-temporal level differences between dysarthric, elderly
and normal speech that systematically manifest in articulatory imprecision,
decreased volume and clarity, slower speaking rates and increased dysfluencies,
novel spectrotemporal subspace basis deep embedding features derived using SVD
speech spectrum decomposition are proposed in this paper to facilitate
auxiliary feature based speaker adaptation of state-of-the-art hybrid DNN/TDNN
and end-to-end Conformer speech recognition systems. Experiments were conducted
on four tasks: the English UASpeech and TORGO dysarthric speech corpora; the
English DementiaBank Pitt and Cantonese JCCOCC MoCA elderly speech datasets.
The proposed spectro-temporal deep feature adapted systems outperformed
baseline i-Vector and xVector adaptation by up to 2.63% absolute (8.63%
relative) reduction in word error rate (WER). Consistent performance
improvements were retained after model based speaker adaptation using learning
hidden unit contributions (LHUC) was further applied. The best speaker adapted
system using the proposed spectral basis embedding features produced the lowest
published WER of 25.05% on the UASpeech test set of 16 dysarthric speakers.
