Towards Robust Waveform-Based Acoustic Models
  We propose an approach for learning robust acoustic models in adverse
environments, characterized by a significant mismatch between training and test
conditions. This problem is of paramount importance for the deployment of
speech recognition systems that need to perform well in unseen environments.
Our approach is an instance of vicinal risk minimization, which aims to improve
risk estimates during training by replacing the delta functions that define the
empirical density over the input space with an approximation of the marginal
population density in the vicinity of the training samples. More specifically,
we assume that local neighborhoods centered at training samples can be
approximated using a mixture of Gaussians, and demonstrate theoretically that
this can incorporate robust inductive bias into the learning process. We
characterize the individual mixture components implicitly via data augmentation
schemes, designed to address common sources of spurious correlations in
acoustic models. To avoid potential confounding effects on robustness due to
information loss, which has been associated with standard feature extraction
techniques (e.g., FBANK and MFCC features), we focus our evaluation on the
waveform-based setting. Our empirical results show that the proposed approach
can generalize to unseen noise conditions, with 150% relative improvement in
out-of-distribution generalization compared to training using the standard risk
minimization principle. Moreover, the results demonstrate competitive
performance relative to models learned using a training sample designed to
match the acoustic conditions characteristic of test utterances (i.e., optimal
vicinal densities).
