Music Source Separation via Stem Discovery

2026-09-25

AI Summary

This study addresses the operational complexity of existing audio-query-based source separation methods, which rely on manually provided, precisely matched feature exemplars. To overcome this limitation, we propose MuS3D, a framework that achieves automated music source separation without manual queries by leveraging audio encoding and an iterative source discovery algorithm to autonomously identify active sound sources within mixed audio. Innovatively, this approach replaces the conventional manual query paradigm with a discovery-based iterative mechanism, preserving the flexibility of audio-query interfaces while significantly reducing user burden. Experimental results demonstrate that MuS3D performs comparably to manual-query baselines and surpasses text-guided models, effectively validating the superiority of the proposed automated architecture.

Abstract

Music source separation (MSS) methods aim to extract stems from music mixtures, which is important, for example, in karaoke, music remixing, and pedagogical applications. While earlier research on MSS systems has been dominated by models targeting narrow sets of general stems, there have recently been attempts to support broader source definitions. One of these methods uses audio queries to provide direct and descriptive control over the desired separation targets based on the sound itself. However, query-based separation remains cumbersome due to the need to provide audio examples with features matching the sources contained within mixtures. This paper proposes Music Source Separation via Stem Discovery (MuS3D), a query-based source separation framework that iteratively discovers active sources from the mixture. On correctly detected sources, our model matches manually queried baselines and surpasses state-of-the-art text-based models. Subjective evaluation indicates encoding artifacts as the limiting factor in current generative separation. The findings suggest that audio-based query representations offer an effective and automatable interface for source separation.

Problem

• Music Source Separation
• Stem Discovery
• Query-based Separation
• Audio Query

Innovation

• Music Source Separation
• Stem Discovery
• Query-based Separation
• Audio Representation
• Generative Separation