Deep Learning-based Non-Intrusive Multi-Objective Speech Assessment
  Model with Cross-Domain Features
  In this study, we propose a cross-domain multi-objective speech assessment
model called MOSA-Net, which can estimate multiple speech assessment metrics
simultaneously. Experimental results show that MOSA-Net can improve the linear
correlation coefficient (LCC) by 0.026 (0.990 vs 0.964 in seen noise
environments) and 0.012 (0.969 vs 0.957 in unseen noise environments) in PESQ
prediction, compared to Quality-Net, an existing single-task model for PESQ
prediction, and improve LCC by 0.021 (0.985 vs 0.964 in seen noise
environments) and 0.047 (0.836 vs 0.789 in unseen noise environments) in STOI
prediction, compared to STOI-Net (based on CRNN), an existing single-task model
for STOI prediction. Moreover, MOSA-Net, originally trained to assess objective
scores, can be used as a pre-trained model to be effectively adapted to an
assessment model for predicting subjective quality and intelligibility scores
with a limited amount of training data. Experimental results show that MOSA-Net
can improve LCC by 0.018 (0.805 vs 0.787) in MOS prediction, compared to
MOS-SSL, a strong single-task model for MOS prediction. In light of the
confirmed prediction capability, we further adopt the latent representations of
MOSA-Net to guide the speech enhancement (SE) process and derive a
quality-intelligibility (QI)-aware SE (QIA-SE) approach accordingly.
Experimental results show that QIA-SE provides superior enhancement performance
compared with the baseline SE system in terms of objective evaluation metrics
and qualitative evaluation test. For example, QIA-SE can improve PESQ by 0.301
(2.953 vs 2.652 in seen noise environments) and 0.18 (2.658 vs 2.478 in unseen
noise environments) over a CNN-based baseline SE model.
