TF-MossFormer: Integrating Convolution Gated Local-Global Attentions for Enhanced Time-Frequency Domain Monaural Speech Separation

2026-07-23

AI Summary

Monaural speech separation struggles to simultaneously preserve local continuity and capture long-range dependencies in speech signals. To address this challenge, this work proposes TF-MossFormer, a novel architecture that jointly models the two-dimensional spectro-temporal structure in the time–frequency domain, thereby overcoming the limitations of conventional time-domain chunking strategies. The core of the method is a content-aware sliding window attention mechanism that dynamically adjusts its receptive field to effectively integrate local and global contextual information, complemented by convolutional gating to enhance feature selection and information flow. Evaluated on the WSJ0-2Mix dataset, the model achieves SI-SDRi scores of 22.6 dB, 24.0 dB, and 24.4 dB with parameter counts of 5.9M, 16.9M, and 25.4M, respectively, outperforming existing state-of-the-art approaches.

Abstract

Transformers with global attention capture long-range dependencies but can miss the fine-grained local continuity crucial for speech separation. We propose TF-MossFormer, a time-frequency transformer that combines local and global attention to jointly model short- and long-range contexts for monaural speech separation. At its core is a content-aware sliding-window attention mechanism that dynamically adapts receptive fields for stronger local interactions, avoiding the rigidity of static convolutions. Unlike time-domain chunk-based methods, TF-MossFormer leverages the 2D spectrogram to model structure along both time and frequency axes. Convolutional gating between attention layers further improves feature selection and information flow. TF-MossFormer achieves SI-SDRi of 22.6, 24.0, and 24.4 dB on WSJ0-2Mix with 5.9M, 16.9M, and 25.4M parameters, respectively, outperforming prior approaches.

Problem

• monaural speech separation
• local continuity
• global attention
• time-frequency domain
• fine-grained structure

Innovation

• local-global attention
• content-aware sliding-window
• time-frequency domain
• convolutional gating
• monaural speech separation