On Architectures and Training for Raw Waveform Feature Extraction in ASR
  With the success of neural network based modeling in automatic speech
recognition (ASR), many studies investigated acoustic modeling and learning of
feature extractors directly based on the raw waveform. Recently, one line of
research has focused on unsupervised pre-training of feature extractors on
audio-only data to improve downstream ASR performance. In this work, we
investigate the usefulness of one of these front-end frameworks, namely
wav2vec, in a setting without additional untranscribed data for hybrid ASR
systems. We compare this framework both to the manually defined standard
Gammatone feature set, as well as to features extracted as part of the acoustic
model of an ASR system trained supervised. We study the benefits of using the
pre-trained feature extractor and explore how to additionally exploit an
existing acoustic model trained with different features. Finally, we
systematically examine combinations of the described features in order to
further advance the performance.
