AnyBand: Unified Multi-Bandwidth Speech Extension via Frequency-Aware In-Context Spectral Infilling

2026-08-01

AI Summary

This work addresses the limitation of conventional speech bandwidth extension methods, which require separate models trained for specific cutoff frequencies and thus struggle with continuously varying input bandwidths in real-world scenarios. The authors propose AnyBand, a unified framework that reformulates bandwidth extension as a context-aware spectral inpainting task, leveraging the available low-frequency content as a spectral prompt to guide high-frequency generation. For the first time, this approach enables a single model to handle arbitrary cutoff frequencies without retraining. AnyBand integrates missing-band conditional flow matching, a frequency-aware diffusion Transformer, and a multi-view adversarial refinement mechanism to effectively capture cross-band interactions and long-range dependencies. Experiments demonstrate that AnyBand significantly improves spectral reconstruction quality across diverse datasets and bandwidth configurations, achieving superior perceptual performance under both standard and irregular cutoff frequencies.

Abstract

Bandwidth extension (BWE) aims to recover missing high-frequency content from band-limited speech. Existing methods often formulate BWE as a fixed or predefined bandwidth conversion problem, potentially requiring cutoff-specific models or retraining when the input bandwidth changes. This assumption limits their applicability to practical scenarios where speech may arrive with diverse cutoff frequencies. We propose AnyBand, a unified BWE framework that recasts bandwidth extension as in-context spectral infilling. Motivated by prompt-based zero-shot speech generation, AnyBand conditions high-frequency generation on the observed low-frequency spectrum, using the available band as a frequency-domain prompt that conveys content, speaker, prosodic, and spectral-envelope cues. This formulation enables a single model to perform cutoff-conditioned generation over a continuous range of input bandwidths. AnyBand is trained with missing-band conditional flow matching and an Easy-to-Balanced cutoff curriculum over continuously sampled cutoff frequencies. To better exploit the spectral prompt, we introduce a frequency-aware Diffusion Transformer that models cross-frequency interactions and long-range temporal dependencies, followed by a physically motivated multi-view adversarial refinement stage to enhance spectral realism, envelope coherence, and harmonic consistency. Experiments on multiple datasets and bandwidth settings show that AnyBand consistently improves spectral reconstruction over existing baselines while achieving competitive perceptual quality across both standard and irregular input cutoffs. Audio samples are available.

Problem

• bandwidth extension
• speech processing
• variable cutoff frequency
• spectral infilling
• multi-bandwidth speech

Innovation

• bandwidth extension
• in-context learning
• spectral infilling
• frequency-aware diffusion
• multi-view adversarial refinement