Exposure Diffusion: HDR Image Generation by Consistent LDR denoising

2024-05-23

arXiv.org

AI Summary

Existing HDR image synthesis methods are hindered by the scarcity of large-scale HDR training data and high computational resource demands. Method: This paper proposes a novel paradigm that synthesizes high-quality HDR images without any HDR training data, leveraging only multiple black-box pre-trained LDR diffusion models in collaboration. It introduces exposure consistency constraints into the diffusion sampling process—achieved via parallel multi-model denoising, exposure-aware latent variable coupling, and a physics-based exposure model regularizer—to emulate conventional HDR bracketing. The framework supports unconditional generation, class-conditional generation, and LDR-to-HDR restoration. Results: Experiments demonstrate that our zero-HDR-training approach significantly enhances dynamic range and perceptual realism, outperforming state-of-the-art HDR synthesis methods while eliminating reliance on HDR ground truth or specialized training.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

We demonstrate generating high-dynamic range (HDR) images using the concerted action of multiple black-box, pre-trained low-dynamic range (LDR) image diffusion models. Common diffusion models are not HDR as, first, there is no sufficiently large HDR image dataset available to re-train them, and second, even if it was, re-training such models is impossible for most compute budgets. Instead, we seek inspiration from the HDR image capture literature that traditionally fuses sets of LDR images, called"brackets", to produce a single HDR image. We operate multiple denoising processes to generate multiple LDR brackets that together form a valid HDR result. To this end, we introduce an exposure consistency term into the diffusion process to couple the brackets such that they agree across the exposure range they share. We demonstrate HDR versions of state-of-the-art unconditional and conditional as well as restoration-type (LDR2HDR) generative modeling.

Problem

• Generate HDR images using pre-trained LDR diffusion models
• Overcome lack of large HDR datasets and high training costs
• Ensure consistency across LDR brackets for valid HDR results

Innovation

• Uses multiple pre-trained LDR diffusion models
• Introduces consistency term in diffusion process
• Generates HDR images from LDR exposure brackets