CogniEdit: Dense Gradient Flow Optimization for Fine-Grained Image Editing

2025-12-15

AI Summary

Existing instruction-driven image editing methods struggle to precisely control fine-grained attributes—such as color, spatial position, and object count—and rely on sparse, single-step optimization feedback, lacking trajectory-level control. To address this, we propose a Dense Gradient Flow Optimization framework that, for the first time, backpropagates reward signals continuously along the diffusion denoising trajectory. Our method integrates multimodal large-model-based instruction parsing, dynamic token-focused relocalization, and a novel Dense Group Relative Policy Optimization (GRPO) strategy, enabling end-to-end trajectory-level supervision. Evaluated on multiple benchmarks, our approach achieves state-of-the-art performance, significantly improving fine-grained instruction adherence while preserving high visual fidelity and original image editability.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Search and Optimization: Learning to Search
• Natural Language Processing: Learning & Optimization for NLP

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

Instruction-based image editing with diffusion models has achieved impressive results, yet existing methods strug- gle with fine-grained instructions specifying precise attributes such as colors, positions, and quantities. While recent approaches employ Group Relative Policy Optimization (GRPO) for alignment, they optimize only at individual sampling steps, providing sparse feedback that limits trajectory-level control. We propose a unified framework CogniEdit, combining multi-modal reasoning with dense reward optimization that propagates gradients across con- secutive denoising steps, enabling trajectory-level gradient flow through the sampling process. Our method comprises three components: (1) Multi-modal Large Language Models for decomposing complex instructions into actionable directives, (2) Dynamic Token Focus Relocation that adaptively emphasizes fine-grained attributes, and (3) Dense GRPO-based optimization that propagates gradients across consecutive steps for trajectory-level supervision. Extensive experiments on benchmark datasets demonstrate that our CogniEdit achieves state-of-the-art performance in balancing fine-grained instruction following with visual quality and editability preservation

Problem

• Optimizes fine-grained image editing with precise attribute control
• Propagates dense gradients across denoising steps for trajectory-level supervision
• Decomposes complex instructions using multi-modal reasoning for actionable directives

Innovation

• Multi-modal reasoning for decomposing complex instructions
• Dynamic token focus relocation for fine-grained attributes
• Dense gradient flow optimization across denoising steps