Non-Intrusive Binaural Speech Intelligibility Prediction from Discrete
  Latent Representations
  Non-intrusive speech intelligibility (SI) prediction from binaural signals is
useful in many applications. However, most existing signal-based measures are
designed to be applied to single-channel signals. Measures specifically
designed to take into account the binaural properties of the signal are often
intrusive - characterised by requiring access to a clean speech signal - and
typically rely on combining both channels into a single-channel signal before
making predictions. This paper proposes a non-intrusive SI measure that
computes features from a binaural input signal using a combination of vector
quantization (VQ) and contrastive predictive coding (CPC) methods. VQ-CPC
feature extraction does not rely on any model of the auditory system and is
instead trained to maximise the mutual information between the input signal and
output features. The computed VQ-CPC features are input to a predicting
function parameterized by a neural network. Two predicting functions are
considered in this paper. Both feature extractor and predicting functions are
trained on simulated binaural signals with isotropic noise. They are tested on
simulated signals with isotropic and real noise. For all signals, the ground
truth scores are the (intrusive) deterministic binaural STOI. Results are
presented in terms of correlations and MSE and demonstrate that VQ-CPC features
are able to capture information relevant to modelling SI and outperform all the
considered benchmarks - even when evaluating on data comprising of different
noise field types.
