Perceptual Quality Loss or Loss of Perceptual Quality?

2026-09-28

AI Summary

This study addresses the issue that over-reliance on PESQ loss in speech enhancement training leads to inconsistencies between objective metrics and subjective perceptual quality, alongside risks of cross-dataset generalization failure. To investigate this, we systematically evaluate the impact of various auxiliary PESQ losses on model performance through comprehensive validation combining deep learning frameworks, standard objective metric evaluations, and formal subjective listening tests. Our findings reveal a dominant bias introduced by PESQ within composite loss functions; notably, experimental results demonstrate that models trained without PESQ loss are actually preferred by listeners. This work highlights the necessity of a complete joint subjective-objective evaluation pipeline and provides critical guidance for loss function design in speech enhancement.

Abstract

Contemporary deep speech enhancement (SE) models are often trained with specific auxiliary terms in the loss function as a way to improve their performance in terms of perceptual metrics. Nevertheless, a higher score on a perceptual metric does not necessarily correlate with an improved listening experience. Through objective and subjective experiments, we assess the performance of SE models trained with two different types of auxiliary PESQ loss terms. The numerical evaluation on a suite of standard metrics suggests that, while models optimized for PESQ naturally obtain higher PESQ scores in the test set, for most other metrics the scores do not significantly change. In some cases, the PESQ loss even results in worse PESQ scores on mismatched data. A formal listening experiment reveals that the models without a PESQ loss were generally preferred over models that include it, across all settings. Finally, we analyze the relative importance of PESQ in the composite metrics CSIG, CBAK and COVL, and find that PESQ dominates all of them. Our study highlights the perils of over-reliance on PESQ and stresses the importance of a complete evaluation procedure for SE.

Problem

• Speech Enhancement
• Perceptual Quality
• PESQ
• Loss Function
• Evaluation Metrics

Innovation

• Speech Enhancement
• Perceptual Quality
• PESQ Loss
• Subjective Evaluation
• Composite Metrics