End-to-End Multi-Task Learning for Adjustable Joint Noise Reduction and Hearing Loss Compensation

2026-03-20

AI Summary

This study addresses the challenge of jointly achieving adjustable noise suppression and personalized hearing loss compensation within a unified model. The authors propose an end-to-end multi-task learning framework that leverages a single deep neural network to simultaneously optimize both tasks, employing two time-frequency masks to enable independent adjustment of each function’s intensity during inference. A differentiable auditory model is innovatively integrated into the training pipeline, facilitating end-to-end optimization while allowing personalization based on individual audiograms without requiring retraining. Experimental results demonstrate that the proposed method outperforms both single-task models and cascaded dual-model approaches in objective metrics, achieves hearing compensation performance comparable to conventional hearing aid prescriptions, and offers flexible real-time adjustability.

Technology Category

• Machine Learning: Multimodal Learning
• Natural Language Processing: Learning & Optimization for NLP
• Cognitive Modeling & Cognitive Systems: Adaptive Behavior

Application Category

• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

A multi-task learning framework is proposed for optimizing a single deep neural network (DNN) for joint noise reduction (NR) and hearing loss compensation (HLC). A distinct training objective is defined for each task, and the DNN predicts two time-frequency masks. During inference, the amounts of NR and HLC can be adjusted independently by exponentiating each mask before combining them. In contrast to recent approaches that rely on training an auditory-model emulator to define a differentiable training objective, we propose an auditory model that is inherently differentiable, thus allowing end-to-end optimization. The audiogram is provided as an input to the DNN, thereby enabling listener-specific personalization without the need for retraining. Results show that the proposed approach not only allows adjusting the amounts of NR and HLC individually, but also improves objective metrics compared to optimizing a single training objective. It also outperforms a cascade of two DNNs that were separately trained for NR and HLC, and shows competitive HLC performance compared to a traditional hearing-aid prescription. To the best of our knowledge, this is the first study that uses an auditory model to train a single DNN for both NR and HLC across a wide range of listener profiles.

Problem

• noise reduction
• hearing loss compensation
• multi-task learning
• auditory model
• personalization

Innovation

• multi-task learning
• differentiable auditory model
• end-to-end optimization
• hearing loss compensation
• adjustable noise reduction