X$^3$-OPD: Distilling Reasoning into Large Audio-Language Models via On-Policy Alignment

2026-07-23

AI Summary

This work addresses the significant gap in deep logical reasoning capabilities between current large audio-language models and their text-based counterparts, primarily attributed to the scarcity of high-quality reasoning data in the audio domain. To bridge this gap, the authors propose the X³-OPD framework, which extends cross-modal distillation to non-transcribable audio content—including non-linguistic events, prosody, and conversational context—by leveraging policy-aligned distillation to transfer reasoning abilities from a text-based teacher model to an audio student model. The framework is supported by a tripartite symmetric corpus encompassing spoken-text reasoning, complex acoustic event reasoning, and dialogue reasoning enriched with paralinguistic cues, thereby enabling diverse auditory reasoning scenarios. Experiments demonstrate substantial improvements in grounded audio reasoning and chain-of-thought quality across benchmarks such as MMSU, MMAU, BIG-Bench Audio, and MMAR, while preserving performance under domain transfer.

Abstract

While large audio-language models have achieved remarkable progress in auditory perception, they still lag behind text-based large language models in deep logical reasoning, primarily due to the scarcity of high-quality audio reasoning data. To bridge this gap, we propose X$^3$-OPD, a cross-modal on-policy distillation framework that transfers reasoning capabilities from a powerful text teacher to an audio-language student. During training, the student generates reasoning trajectories conditioned on its own acoustic perception, while the teacher provides token-level guidance using matched textual inputs and verified answers. We further construct a three-tier symmetric corpus covering textual reasoning rendered into speech, audio-event reasoning grounded in complex acoustic scenes, and spoken-dialogue reasoning involving paralinguistic cues. This design extends cross-modal distillation beyond textually recoverable content to reasoning grounded in non-linguistic events, prosody, and conversational context. Experiments on MMSU, MMAU, BIG Bench Audio, and MMAR demonstrate that X$^3$-OPD substantially improves audio-grounded reasoning and chain-of-thought quality while largely preserving the model's existing capabilities under domain shift.

Problem

• audio-language models
• logical reasoning
• reasoning data scarcity
• cross-modal reasoning
• audio-grounded reasoning

Innovation

• on-policy distillation
• audio-language reasoning
• cross-modal alignment
• chain-of-thought
• symmetric corpus