🤖 AI Summary
This study addresses the limitations of large audio-language models in cross-domain representation modeling and multi-task understanding by proposing a unified audio encoder based on a Mixture-of-Experts (MoE) architecture. Methodologically, it introduces a SwiGLU shared expert decoupling network to integrate the strengths of mainstream encoders, alongside a novel two-stage instruction tuning strategy and Task-Specific Data Scaling (TSDS) technique to enhance model generalization. Experimental results demonstrate that the proposed model achieves state-of-the-art performance of 0.802 on the XARES-LLM benchmark and exhibits superior cross-domain understanding capabilities in the Interspeech 2026 Challenge.
📝 Abstract
Large Audio Language Models (LALMs) rely on effective audio encoders for multi-task performance. We introduce UniAE-MoE, a unified audio encoder designed to model cross-domain audio representations and achieve outstanding downstream understanding performance via a Mixture-of-Experts (MoE) architecture. Specifically, we explore mainstream audio encoders and integrate those from Qwen2-Audio and Audio-Flamingo 3, which demonstrate superior downstream capabilities. To facilitate effective model fusion, we improve our encoder using SwiGLU with shared experts to decouple encoder networks, and we further introduce a two-stage instruction-tuning strategy to better adapt the model to diverse downstream tasks. Moreover, we propose the task-specific data scaling (TSDS) technique to enhance \tool's understanding capabilities. On the XARES-LLM benchmark, UniAE-MoE attains a score of 0.802, achieving state-of-the-art performance. It also delivers top-tier performance in the official Interspeech 2026 Audio Encoder Capability Challenge, further demonstrating robust generalization across diverse audio tasks. Together, these results validate the effectiveness of \tool for unified audio understanding across speech, music, and general audio domains.