Transferable Multi-Bit Watermarking Across Frozen Diffusion Models via Latent Consistency Bridges

2026-03-19

AI Summary

This work proposes DiffMark, a novel watermarking method that overcomes the limitations of existing approaches in frozen diffusion models by simultaneously achieving high multi-bit capacity, cross-model transferability, and efficient detection without relying on time-consuming inversion or model-specific fine-tuning. DiffMark injects learnable perturbations at each denoising step to accumulate watermark signals in the latent space, enabling multi-bit watermark detection with a single forward pass and supporting image-level dynamic keys. By leveraging a Latent Consistency Model (LCM) as a differentiable training proxy, the method reduces backpropagation steps from 50 to just 4, dramatically improving computational efficiency. The approach requires no retraining for new models, achieves detection in 16.4 milliseconds—45× faster than prior methods—and demonstrates strong robustness against distortions, regeneration, and adversarial attacks.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Large Multimodal Models (LMMs)
• Search and Optimization: Distributed Search

Application Category

• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

As diffusion models (DMs) enable photorealistic image generation at unprecedented scale, watermarking techniques have become essential for provenance establishment and accountability. Existing methods face challenges: sampling-based approaches operate on frozen models but require costly $N$-step Denoising Diffusion Implicit Models (DDIM) inversion (typically N=50) for zero-bit-only detection; fine-tuning-based methods achieve fast multi-bit extraction but couple the watermark to a specific model checkpoint, requiring retraining for each architecture. We propose DiffMark, a plug-and-play watermarking method that offers three key advantages over existing approaches: single-pass multi-bit detection, per-image key flexibility, and cross-model transferability. Rather than encoding the watermark into the initial noise vector, DiffMark injects a persistent learned perturbation $δ$ at every denoising step of a completely frozen DM. The watermark signal accumulates in the final denoised latent $z_0$ and is recovered in a single forward pass. The central challenge of backpropagating gradients through a frozen UNet without traversing the full denoising chain is addressed by employing Latent Consistency Models (LCM) as a differentiable training bridge. This reduces the number of gradient steps from 50 DDIM to 4 LCM and enables a single-pass detection at 16.4 ms, a 45x speedup over sampling-based methods. Moreover, by this design, the encoder learns to map any runtime secret to a unique perturbation at inference time, providing genuine per-image key flexibility and transferability to unseen diffusion-based architectures without per-model fine-tuning. Although achieving these advantages, DiffMark also maintains competitive watermark robustness against distortion, regeneration, and adversarial attacks.

Problem

• watermarking
• diffusion models
• multi-bit
• transferability
• frozen models

Innovation

• multi-bit watermarking
• frozen diffusion models
• latent consistency models
• cross-model transferability
• single-pass detection