VibeEdit: Image Editing with Canvas Instructions

2026-10-08

AI Summary

This study addresses the challenge of inaccurate localization of similar objects in text-guided image editing by proposing a canvas-instruction-based framework for precise manipulation. By employing spatial markers and concise annotations to directly specify target regions and intended edits, the method enables addition, deletion, modification, and relocation operations without requiring supplementary text prompts. Technically, this work introduces a novel canvas-instruction interaction interface that integrates layer-decoupled conditional encoding, region-weighted supervised fine-tuning, and rubric-guided reinforcement learning, built upon the Qwen-Image-Edit architecture. Experimental results demonstrate that the proposed approach achieves a VLM score of 79.9 and a PSNR of 32.8 dB in non-edited regions on standard benchmarks, significantly outperforming the FireRed baseline and delivering high-fidelity, precise image editing.

Abstract

In text-guided image editing, describing the desired change is often straightforward, but identifying the intended object or region can be cumbersome, especially when several objects look alike. We introduce a new image editing interface that lets users place spatial marks and optional short notes directly on the image. Together, these annotations form a canvas instruction that specifies where to edit and what to change. Our editor, VibeEdit, follows these instructions to perform object addition, removal, replacement, attribute modification, and movement without a separate text prompt. We construct 1.55 million source-target edit pairs with object masks and structured edit descriptions, from which we render canvas instructions during training. We adapt Qwen-Image-Edit with layer-decoupled conditioning that separately encodes source images and canvas instructions for image editing. We train the model with region-weighted supervised fine-tuning, followed by rubric-guided reinforcement learning to improve edit completion, local edit quality, and preservation of unedited regions. We evaluate VibeEdit on an independently constructed, human-curated benchmark of 419 cases emphasizing target selection among similar objects. VibeEdit achieves a VLM rubric score of 79.9 and an outside-region PSNR of 32.8 dB, compared with 67.4 and 24.0 dB for FireRed, the highest-scoring text-instructed baseline in our evaluation.

Problem

• image editing
• text-guided editing
• object selection
• spatial annotation

Innovation

• Canvas Instruction
• Layer-Decoupled Conditioning
• Region-Weighted Supervised Fine-Tuning
• Rubric-Guided Reinforcement Learning
• Text-Guided Image Editing