region proposal

Current text-to-image generation methods face a fundamental trade-off among naturalness, text alignment, and safety, often resulting in localized artifacts (e.g., over-sexualization, violence, implausibility) or global quality degradation. To address this, we propose a fine-grained region-aware fine-tuning paradigm that selectively identifies and rectifies problematic regions—thereby avoiding full-image optimization, which induces reward hacking and cross-dimensional performance collapse. Our method integrates multimodal attention-guided target region localization, masked gradient masking, and a local loss function steered by safety- and plausibility-aware discriminators. Experiments demonstrate a 23.6% improvement in human evaluation scores on key localized metrics (Safety and Plausibility), with no statistically significant degradation in other dimensions; moreover, visual alterations in non-problematic regions remain imperceptible in 92% of samples.

Existing text-driven image region editing methods predominantly rely on local cues while neglecting the synergistic relationship between regional and global semantics, resulting in incoherent edits, abrupt transitions, or semantic imbalance. To address this, we propose a dual-level guidance mechanism: (1) fine-grained textual descriptions at the region level, and (2) context-aware semantic descriptions—generated by large language models—at the scene level, jointly establishing a global-local alignment framework. We further introduce a cross-modal alignment loss and contextually grounded region representations to enable multi-granularity vision-language co-modeling. Our approach significantly improves edit coherence and instruction adherence, effectively mitigating semantic conflicts and structural incoherence across multiple benchmarks. Notably, it is the first method to systematically formalize the cognitive principle of “holistic understanding guiding local modification” in image editing.

Existing text-to-image diffusion models (e.g., Stable Diffusion) struggle to precisely localize and synthesize fine-grained object parts—such as “a panda’s bamboo cane” or “a robot’s glowing joints”—in a zero-shot setting, often resulting in part omission, misplacement, or semantic inconsistency. This work introduces PartCraft, the first fine-grained part-level diffusion localization and synthesis framework that operates without model fine-tuning. Leveraging only a pre-trained diffusion model, PartCraft enables spatially accurate part localization, binary mask generation, localized inpainting, and seamless multi-region compositing—all within the standard denoising process and without introducing auxiliary parameters or training overhead. Extensive qualitative and quantitative evaluations demonstrate that PartCraft significantly improves part fidelity, spatial consistency, and compositional novelty compared to prior part-control methods. It establishes an efficient, general-purpose, zero-shot paradigm for controllable image generation via part-aware editing.

This work addresses the challenge in image editing where fine-grained local structures—such as text or logos—are prone to detail collapse or unintended background alterations. To tackle this, the authors propose a region-specific refinement approach grounded in multimodal diffusion models, featuring a Focus-and-Refine strategy. This strategy reallocates resolution budgets via a crop-and-resize mechanism to concentrate computational resources on the target region, while integrating hybrid mask inpainting and a boundary consistency loss to ensure high-fidelity local reconstruction and strict preservation of non-edited areas. The study also contributes the Refine-30K dataset and the RefineEval benchmark for evaluation. Experimental results demonstrate that the proposed method significantly outperforms existing techniques on RefineEval, achieving near-perfect background retention alongside high-quality detail recovery.

Current instruction-based image editing (IIE) models apply a uniform generative process to both edited and unedited regions, leading to computational redundancy and suboptimal efficiency. To address this, we propose a training-free, adaptive region-aware generation framework—the first to introduce region-differentiated generation. Our method employs diffusion trajectory analysis for adaptive spatial partitioning, designs region-specific instruction key-value (KV) caches and velocity decay caches to accelerate localized denoising iterations, and explicitly models velocity similarity to preserve global coherence. The framework is plug-and-play compatible with mainstream IIE models—including Step1X-Edit, FLUX.1 Kontext, and Qwen-Image-Edit—achieving speedups of 2.57×, 2.41×, and 2.06×, respectively. GPT-4o evaluation confirms that semantic fidelity and visual quality are well preserved. This work establishes a new paradigm for efficient, region-adaptive diffusion-based image editing without architectural or training modifications.

This work addresses the challenge of unifying object localization, segmentation, editing, and description into a single, transparent, reliable, and reproducible image analysis pipeline driven by one natural language instruction. Method: We propose an end-to-end, prompt-driven, multi-stage framework integrating open-vocabulary detection, promptable segmentation, text-conditioned image inpainting, and vision-language description. A transparent multimodal model integration paradigm is introduced, incorporating intermediate result persistence, adaptive threshold tuning, lightweight morphological mask post-processing, and dependency version locking. The system supports both CLI and UI modes, with seed control, comprehensive logging, and parameter serialization. Contribution/Results: Experiments show >90% usability and >85% accuracy for mask generation under word-level prompting; inpainting accounts for 60–75% of total high-end GPU time, validating efficacy while highlighting optimization opportunities.

Existing training-free editing methods struggle to precisely localize editing regions in object replacement tasks, often resulting in residual source objects, constrained generation, or background distortion. This work proposes PC-Edit, a novel framework that introduces, for the first time, a prompt contrast mechanism within multimodal Diffusion Transformers (MM-DiT). By comparing attention maps of image tokens under source and target prompts, PC-Edit directly identifies semantically altered regions and jointly performs region erasure, target generation, and background preservation at each denoising step. Without requiring any training, the method integrates region mask construction with cached key/value feature injection, achieving state-of-the-art performance on both PIE-Bench and the newly introduced EditRegion-Bench, significantly outperforming existing unsupervised approaches in editing accuracy and background fidelity.

Existing diffusion models struggle to precisely control the material properties, object identity, and spatial layout of specific regions in generated images through text prompts alone. To address this limitation, this work proposes an appearance pointer mechanism that introduces a modality-agnostic, region-aware multimodal control interface into diffusion Transformers without requiring retraining. The method employs a region correspondence network to generate appearance pointers and integrates a spatial aggregation mechanism to align textual or image inputs with user-specified masks, thereby guiding the model to apply appropriate appearance features at the correct locations. Seamlessly embedded within existing DiT architectures, the approach supports multi-region descriptions with minimal additional token overhead and achieves competitive or superior performance compared to modality-specific state-of-the-art methods, significantly enhancing the accuracy and flexibility of region-controllable image generation.

This work addresses the high computational cost and difficulty in focusing on task-relevant regions when processing information-dense images—such as infographics and documents—with current large vision-language models, which often generate excessive redundant visual tokens. To mitigate this, the authors propose PinPoint, a two-stage framework that first employs an Instruction-Region Alignment mechanism to precisely localize regions relevant to the given text instruction, and then extracts visual features only from these targeted areas. This approach substantially reduces irrelevant tokens, enhancing both inference efficiency and accuracy. The study also introduces the first annotated dataset specifically designed for instruction-driven region localization and demonstrates state-of-the-art performance on challenging document-based VQA benchmarks, including InfographicVQA.

Existing one-step image editing methods lack explicit spatial control, making it challenging to achieve strong semantic and structurally consistent modifications within user-specified regions. This work proposes a locally adaptive editing framework that leverages a mask-aware mechanism to automatically identify semantically relevant areas and applies adaptive modulation in the latent space to precisely edit only the target region while preserving the rest of the image unchanged. By integrating internal feature-driven editable region discovery, localized latent modulation, and spatial constraints, the method significantly outperforms current one-step approaches on PIE-Bench, striking an effective balance between editing fidelity and computational efficiency. These results underscore the critical role of explicit spatial reasoning in enabling high-quality image editing.