Exposing and Mitigating Neural Codec Vulnerabilities in Audio Deepfake Detection

2026-10-05

AI Summary

This study addresses the severe performance degradation of existing audio deepfake detection models under neural codec compression, which frequently misclassifies bona fide speech as spoofed. To bridge this robustness gap, we construct a dedicated spoofing dataset and propose PCL-NET, a novel method employing XLS-R as a pretrained encoder. Crucially, PCL-NET introduces a pioneering paired consistency learning strategy that effectively decouples codec resynthesis artifacts from genuine spoof-discriminative features. Experimental results demonstrate that the proposed approach significantly reduces the average equal error rate (EER) under codec compression from 28.67% to 12.77%, while maintaining competitive deepfake detection performance in standard scenarios.

Abstract

Existing audio deepfake detection (ADD) datasets and detectors are primarily built for vocoder-based synthesis, evaluated against traditional post-hoc perturbations such as MP3/AAC compression or additive noise, applied independently of generation. However, recent speech synthesizers, particularly ALM-based systems, use neural audio codecs both for compression and as the resynthesis reconstructing waveforms from generated tokens, producing artifacts distinct from post-hoc compression. Neural codecs thus play a dual role: some are designed for pure compression under low-bandwidth communication, while others serve as resynthesis components. Despite this dual role, robustness to codec-based compression, unlike post-hoc compression, remains largely unexplored. We expose this gap, showing that state-of-the-art (SOTA) ADD models degrade drastically on codec-compressed speech; in particular, systems trained on Codec Resynthesized data as a proxy for codec-based generation prove most vulnerable, with legitimately compressed bona fide speech often misclassified as fake. To investigate this, we construct the Audio Neural Codec-Spoof dataset by applying seven neural codec algorithms to existing ADD benchmarks, isolating codec-induced resynthesis artifacts as a controlled proxy for codec-based generation. As baseline mitigation, we propose PCL-NET (Pairwise Consistency Learned Network), fine-tuning a pretrained XLS-R (300M) encoder with a pairwise consistency objective that minimizes the representation distance between an utterance's uncompressed and codec-compressed versions, disentangling codec artifacts from the real-versus-fake decision. As a result, PCL-NET reduces average EER under neural codec compression from 28.67% to 12.77%, while preserving competitive CoSG-based deepfake detection performance. We will also make the dataset publicly available on Hugging Face upon acceptance.

Problem

• Audio Deepfake Detection
• Neural Audio Codec
• Robustness
• Codec Compression Artifacts

Innovation

• Audio Deepfake Detection
• Neural Codec
• Pairwise Consistency Learning
• PCL-NET
• Codec Resynthesis Artifacts