Investigating Codec-Internal Latent Audio Watermarking for Neural Codec Robustness

2026-07-23

AI Summary

This work addresses the vulnerability of traditional audio watermarking schemes to neural audio codecs by proposing a method that embeds 32-bit messages directly into the continuous latent space of a codec-like speech autoencoder, prior to neural decoding. The approach significantly enhances robustness against neural codec transformations through a SEANet-style encoder-decoder architecture, a Conformer-based message embedder, RVQ-guided latent space decomposition, and a latent-domain detector. The system is trained under joint perturbations from both conventional signal processing and neural codecs. Evaluated on 48 kHz speech and tested after perceptual training with EnCodec, the method improves bit accuracy from 78.8% to 95.6% and 97.1% under EnCodec-24k compression, with only minor PESQ degradation to 3.514 and 3.427, respectively.

Abstract

Neural audio codecs are challenging transformations for audio watermarking because they re-encode, quantize, and resynthesize speech. This paper investigates continuous latent-space watermarking for codec robustness. Instead of adding a watermark only to the waveform or spectrogram, we embed a 32-bit message into the continuous latent representation of a codec-like speech autoencoder. The pipeline uses a SEANet-style encoder-decoder, a Conformer-based message embedder, RVQ-guided latent decomposition, and a latent-domain detector trained under signal-processing and neural-codec transformations. Rather than proposing a final universal watermarking baseline, we characterize the trade-offs that appear when the watermark carrier is moved before neural decoding. On 48 kHz speech, EnCodec-aware training improves EnCodec-24k bit accuracy from 78.8% to 95.6% and 97.1%, while PESQ decreases from 3.727 to 3.514 and 3.427.

Problem

• neural audio codec
• audio watermarking
• latent-space watermarking
• codec robustness
• speech autoencoder

Innovation

• latent-space watermarking
• neural audio codec
• RVQ-guided decomposition
• Conformer-based embedder
• codec-robust watermarking