STFTCodec: High-Fidelity Audio Compression through Time-Frequency Domain Representation

2025-03-21

AI Summary

To address the challenge of simultaneously achieving high fidelity, low model capacity, and flexible bit-rate control in neural audio compression, this paper proposes STFTCodec—a novel STFT-based neural audio codec. Methodologically, it introduces unwrapped phase derivatives as auxiliary features for the first time, relaxing strict phase reconstruction constraints while preserving phase-awareness; employs a parallel magnitude/phase dual-branch architecture with compact spectral representation; and enables bit-rate adaptability solely by tuning STFT parameters—without modifying network architecture. Experiments demonstrate that STFTCodec consistently outperforms state-of-the-art waveform- and spectrogram-based codecs across multiple bit rates, yielding substantial improvements in subjective audio quality (MOS increase of 0.4–0.8) while reducing model parameters by 37% and memory footprint by 29%.

Technology Category

• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Machine Learning: Learning on the Edge & Model Compression
• Natural Language Processing: Learning & Optimization for NLP

Application Category

• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets

Abstract

We present STFTCodec, a novel spectral-based neural audio codec that efficiently compresses audio using Short-Time Fourier Transform (STFT). Unlike waveform-based approaches that require large model capacity and substantial memory consumption, this method leverages STFT for compact spectral representation and introduces unwrapped phase derivatives as auxiliary features. Our architecture employs parallel magnitude and phase processing branches enhanced by advanced feature extraction mechanisms. By relaxing strict phase reconstruction constraints while maintaining phase-aware processing, we achieve superior perceptual quality. Experimental results demonstrate that STFTCodec outperforms both waveform-based and spectral-based approaches across multiple bitrates, while offering unique flexibility in compression ratio adjustment through STFT parameter modification without architectural changes.

Problem

• Efficient audio compression using STFT representation
• Reducing model capacity and memory consumption
• Superior perceptual quality with flexible compression adjustment

Innovation

• Uses STFT for compact spectral representation
• Introduces unwrapped phase derivatives as features
• Parallel magnitude and phase processing branches