Voice Quality and Pitch Features in Transformer-Based Speech Recognition
  Jitter and shimmer measurements have shown to be carriers of voice quality
and prosodic information which enhance the performance of tasks like speaker
recognition, diarization or automatic speech recognition (ASR). However, such
features have been seldom used in the context of neural-based ASR, where
spectral features often prevail. In this work, we study the effects of
incorporating voice quality and pitch features altogether and separately to a
Transformer-based ASR model, with the intuition that the attention mechanisms
might exploit latent prosodic traits. For doing so, we propose separated
convolutional front-ends for prosodic and spectral features, showing that this
architectural choice yields better results than simple concatenation of such
pitch and voice quality features to mel-spectrogram filterbanks. Furthermore,
we find mean Word Error Rate relative reductions of up to 5.6% with the
LibriSpeech benchmark. Such findings motivate further research on the
application of prosody knowledge for increasing the robustness of
Transformer-based ASR.
