Decoupled Residual Denoising Diffusion Models for Unified and Data Efficient Image-to-Image Translation

2026-05-31

AI Summary

This work addresses the challenge in existing diffusion models for image-to-image translation, where achieving both task unification and data efficiency remains difficult, and the implicit domain alignment induced by noise injection is prematurely lost during coupled denoising. To overcome this, the authors propose the Decoupled Residual Denoising Diffusion (DRDD) model, which separates the diffusion process into two stages: first, stochastic noise diffusion is applied to achieve domain alignment and manifold elevation using only unpaired target-domain images; second, deterministic residual diffusion is performed within the fixed noise domain to learn semantic mappings. This approach is the first to explicitly reveal and effectively preserve the domain alignment capability of noise, substantially improving data efficiency and cross-task consistency. The method enables robust and generalizable image translation even with limited paired data, with both theoretical analysis and extensive experiments confirming its effectiveness and broad applicability.

Abstract

We propose Decoupled Residual Denoising Diffusion models (DRDD) for unified and data-efficient image-to-image (I2I) translation. While diffusion models have advanced I2I translation in terms of quality and diversity, we uncover a previously under-explored property in diffusion models. Crucially, beyond its conventional role of manifold lifting (i.e., moving data off low-dimensional manifolds), injecting Gaussian noise facilitates domain harmonization by implicitly aligning feature distributions across domains, a property particularly advantageous for unified I2I translation. However, existing diffusion models prematurely erode this harmonization effect, as noise and residuals are simultaneously removed in a single coupled diffusion process. To address this, DRDD decouples the diffusion process into two sequential and independent diffusion stages: (1) a stochastic noise diffusion for domain harmonization and manifold lifting, and (2) a deterministic residual diffusion that learns the core semantic mapping entirely within the fixed-noise domain. This decoupling preserves harmonization and manifold lifting effects throughout the transformation, substantially simplifying the learning of unified mappings across diverse tasks and domains. Notably, the noise diffusion stage is trained exclusively on abundant, unpaired target-domain images, greatly improving data efficiency. Comprehensive theoretical and empirical analysis demonstrates that DRDD is broadly compatible with mainstream diffusion models and consistently delivers robust, unified I2I translation, even under limited paired data. Our code is available at https://github.com/HKU-HealthAI/DRDD.

Problem

• image-to-image translation
• diffusion models
• domain harmonization
• data efficiency
• manifold lifting

Innovation

• Decoupled Diffusion
• Domain Harmonization
• Manifold Lifting
• Data-Efficient I2I Translation
• Residual Denoising