IEA: Amateur-Friendly Conversational Image Editing Agent via Three Stages of Multitask Alignment

2026-06-06

AI Summary

Existing image editing methods often struggle to accurately interpret the intent of non-expert users and frequently produce distorted or uninterpretable results. This work proposes IEA, a conversational image editing agent that jointly models tool invocation, interpretable editing trajectories, and user intent summarization. Through a three-stage multi-task alignment training pipeline—comprising supervised fine-tuning, GRPO optimization, and synthetic data fine-tuning—IEA performs high-fidelity edits within an interpretable action space defined by 16 parameterized tools. Experimental results demonstrate that IEA outperforms strong baselines in both pixel-level editing error and ROUGE-L scores for intent summarization. User studies further confirm that IEA significantly surpasses existing tool-calling and generative approaches in instruction following and perceived output quality.

Abstract

Current image editing software often hinges on fixed filters or expert tuning, leaving a gap between amateur users' intent and outcomes. Creations by generative models may contain artifacts, implausible details, or stylistic drift away from photorealism and offer little insight into why an edit was made. We propose IEA, a conversational Image Editing Agent that learns to operate parameterized tools in an explicit, interpretable action space. IEA is trained via a three-stage multitask pipeline: (1) SFT on distilled expert edits, (2) GRPO with rewards for likeness improvement, tool usefulness, and intent summarization, and (3) large-scale synthetic fine-tuning to jointly master image editing, refinement, and user intent summarization. By manipulating 16 editing tools step by step, IEA produces transparent edit traces that can be inspected and debugged. In quantitative experiments, it attains a lower pixel distance on the edit task and a higher ROUGE-L on the summary task than strong baselines. In user studies, it ranks best among tool-calling methods for instruction following while surpassing generative methods in overall perceptual quality. Our results validate interpretable, tool-centric VLMs as a reliable path to human instruction-guided image retouching.

Problem

• image editing
• amateur users
• intent-outcome gap
• photorealism
• interpretability

Innovation

• conversational image editing
• multitask alignment
• interpretable action space
• tool-centric VLM
• GRPO fine-tuning