EDSep: An Effective Diffusion-Based Method for Speech Source Separation

2025-01-27

AI Summary

Existing single-channel speech separation diffusion methods suffer from slow sampling and suboptimal separation quality. This paper proposes EDSep, an end-to-end diffusion model grounded in stochastic differential equation (SDE)-based score matching. Its core contributions are twofold: (i) the first denoising network architecture specifically designed for speech separation, and (ii) a novel stochastic sampling strategy that jointly optimizes score estimation accuracy and numerical efficiency of SDE inversion. This design significantly accelerates both training convergence and inference while improving source signal fidelity. Evaluated on three standard benchmarks—WSJ0-2mix, LRS2-2mix, and VoxCeleb2-2mix—EDSep achieves state-of-the-art performance in SI-SNR improvement (SI-SNRi), consistently outperforming existing diffusion-based and discriminative approaches with average gains of 1.2–2.3 dB, thereby demonstrating superior efficiency and effectiveness.

Technology Category

• Natural Language Processing: Speech
• Computer Vision: Diffusion Models for Vision
• Machine Learning: Mixture of Experts (MoE)

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search

Abstract

Generative models have attracted considerable attention for speech separation tasks, and among these, diffusion-based methods are being explored. Despite the notable success of diffusion techniques in generation tasks, their adaptation to speech separation has encountered challenges, notably slow convergence and suboptimal separation outcomes. To address these issues and enhance the efficacy of diffusion-based speech separation, we introduce EDSep, a novel single-channel method grounded in score matching via stochastic differential equation (SDE). This method enhances generative modeling for speech source separation by optimizing training and sampling efficiency. Specifically, a novel denoiser function is proposed to approximate data distributions, which obtains ideal denoiser outputs. Additionally, a stochastic sampler is carefully designed to resolve the reverse SDE during the sampling process, gradually separating speech from mixtures. Extensive experiments on databases such as WSJ0-2mix, LRS2-2mix, and VoxCeleb2-2mix demonstrate our proposed method's superior performance over existing diffusion and discriminative models, validating its efficacy.

Problem

• Speech Separation
• Diffusion Methods
• Efficiency and Clarity

Innovation

• Diffusion Principle
• Denoiser Tool
• Random Differential Equations