EchoDistill:Alignment Noisy-to-Clean Self-Distillation for Robust Audio LLMs

2026-05-11

arXiv.org

AI Summary

Large audio language models often lose critical acoustic evidence under severe noise, leading to unreliable responses. This work proposes EchoDistill, a novel noisy-to-clean self-distillation framework that leverages clean audio as privileged information to guide a teacher model. The student model is optimized through masked token distillation and task-gated consistency shaping, while the backbone network remains frozen to ensure zero additional inference overhead. Experimental results demonstrate that this approach improves average accuracy by 1.63% under strong noise conditions at −10 dB, with Qwen2.5-Omni achieving 62.94%. These findings confirm that EchoDistill effectively enhances robustness against acoustic degradation without incurring extra computational costs during inference.

Abstract

Audio Large Language Models (ALLMs) are highly vulnerable to real-world noise, which often induces severe semantic drift and hallucinations. Existing robustness methods primarily rely on waveform-level acoustic enhancement, answer-level supervision, or the internal suppression of noise representations. To address these issues, we propose echodistill, an alignment-based noisy-to-clean self-distillation framework. Echodistill leverages a frozen clean-audio teacher to provide semantic references for an inference-time noisy-audio student. Specifically, the student samples candidate responses under noisy conditions to expose its test-time behavior. These trajectories are then optimized via group-relative policy optimization (GRPO), where the token-level consistency with the teacher acts as a reward bonus. By aligning the noisy student's candidate responses with clean semantic evidence, and applying audio-aware reward shaping, our method encourages reasoning trajectories that are both correct and genuinely acoustically grounded. Echodistill significantly improves the semantic reliability and task performance of Audio LLMs under complex noise, without introducing any additional inference costs. Extensive experiments show that: (I) Compared with the strongest baseline, echodistill achieves average improvements of 4.18\%$\uparrow$ in GSR under strong noise. (II) Ablation results on Qwen-Omni further show that echodistill improves over the GRPO-only variant by 3.02\%$\uparrow$ in Acc, 3.89\%$\uparrow$ in Noisy, and 4.53\%$\uparrow$ in GSR on average. Our codes are available at https://anonymous.4open.science/r/echodistill-10DE.

Problem

• Large Audio Language Models
• acoustic noise
• robustness
• noisy input

Innovation

• Self-Distillation
• Large Audio Language Models
• Noise Robustness
• Masked Response-Token Distillation
• Post-Training