Problem
Research questions and friction points this paper is trying to address.
instruction-based image editing
text-to-image generation
denoising conditioning
unified models
task switching
When Text-to-Image Helps Editing: The Effects of Conditioning During Denoising
🤖 AI Summary
This study addresses the limitation that unified image editing models suffer from constrained editing quality due to persistent source-image conditioning. We identify an attention decay phenomenon during pure editing processes and accordingly propose a task-switching mechanism. This method introduces, for the first time within unified editors, a training paradigm that dynamically alternates between text-to-image (T2I) generation and editing tasks. By leveraging the denoising characteristics of diffusion models, T2I capabilities are incorporated at specific stages to assist editing, effectively balancing generative quality with content preservation. Experiments demonstrate that this technique significantly improves editing quality across multiple benchmarks while maintaining perceptual consistency comparable to that of pure editing approaches.
📝 Abstract
Unified models are trained for both instruction-based image editing and text-to-image (T2I) generation, but standard editing pipelines keep source-image conditioning throughout denoising. We ask whether editing can benefit from T2I, and study how the effects of conditioning vary across edits and denoising stages. In pure editing, source attention declines for some edits over the sampling trajectory. This observation led us to task switching, which lets the model draw on its T2I capabilities. Across three unified editors and four benchmarks, switching to the T2I task for bounded intervals improves edit quality, while mean perceptual preservation remains close to pure editing across all three models. Unified editors therefore benefit from using both conditioning modes they are trained for, and the timing of the switch sets the balance between quality and preservation.
Innovation
Methods, ideas, or system contributions that make the work stand out.
Task Switching
Text-to-Image Generation
Instruction-based Image Editing
Denoising Conditioning
Unified Models
🔎 Similar Papers
No similar papers found.
