Problem
Research questions and friction points this paper is trying to address.
Physically Based Rendering
Latent Space
Image Diffusion Models
Feature Space
Light Transport Simulation
Physically Based Rendering in the Latent Space
🤖 AI Summary
本文通过在生成模型的潜在空间中引入基于物理的渲染方法，解决了图像扩散模型难以控制的问题，实现了物理引导的内容生成。
📝 Abstract
Image diffusion models have shown impressive image generation capabilities but are often hard to control, in contrast to classical computer graphics pipelines such as physically based rendering. However, we observe that there is a bridge between light transport phenomena and the distribution of latent space values produced by such models. Thus, we introduce physically based rendering in the feature space learned by the variational autoencoders in generative models, enabling light transport simulation in the latent space. This allows us to leverage physically based rendering techniques to output latent maps for physically guided content generation. We propose modifications to the rendering equation, which, when paired with a differentiable renderer, can yield an optimal set of scene parameters that require only minimal refinement to accurately render into the pretrained latent space. We train our method on a single rendered image, and then demonstrate the generalization of the method to scene geometry changes, lighting changes, and camera view changes.
Innovation
Methods, ideas, or system contributions that make the work stand out.
Physically Based Rendering
Latent Space
Differentiable Renderer
Light Transport Simulation
Generative Models
🔎 Similar Papers
No similar papers found.
