MT-EditFlow: Reinforcement Learning for Multi-Turn Image Editing with Flow Matching

2026-06-01

AI Summary

Existing single-turn image editing models often fail in multi-turn interactions due to error accumulation, limiting their ability to support iterative user edits. This work proposes MT-EditFlow, a novel framework that integrates flow matching with reinforcement learning (GRPO/NFT) to enhance the global success rate of sequential editing through multi-turn perspective modeling and a multi-reward optimization mechanism. The key innovation lies in broadcasting aggregated advantage signals across the entire editing trajectory, effectively aligning local editing actions with the global objective. Furthermore, the method systematically designs low-bias, low-variance reward functions to stabilize training. Evaluated on FLUX.1-Kontext-dev, MT-EditFlow achieves a 6.85-point improvement in third-turn editing performance over strong baselines such as Qwen-Image-Edit, while significantly reducing exposure bias and maintaining high marginal success rates.

Abstract

Recent breakthroughs in instruction-based image editing have captured significant attention, as models are now capable of handling real-world editing demands with the practicality required by everyday users. However, editing models trained primarily for single-turn edits often break down in multi-turn editing--the natural interactive setting where a user iteratively refines an image based on the model's own previous outputs. This failure stems from the all-or-nothing requirement, where a single failed turn compromises the entire sequence, and error propagation, where exposure bias leads to compounding editing errors. To address these challenges, we introduce MT-EditFlow, a flow-matching reinforcement learning framework designed to optimize reward signals for sequential image editing. MT-EditFlow integrates a multi-turn perspective with a multi-reward formulation to provide a unified structure applicable to both GRPO and NFT-based reinforcement learning methods. We systematically analyze and optimize the reward signal by investigating effective scoring strategies for turn-level aggregation, VLM reasoning modes to trade off reward bias and variance, and advantage fusion levels to prevent reward hacking. Our findings reveal that broadcasting the aggregated advantage across the entire editing trajectory effectively bridges the gap between local planning and global multi-turn task success. Extensive experiments demonstrate that MT-EditFlow significantly improves performance across diverse base models. Notably, it boosts FLUX.1-Kontext-dev by 6.85 points in turn-3 overall performance, surpassing state-of-the-art open-source models such as Qwen-Image-Edit. By maintaining high marginal success rates and reducing exposure bias, MT-EditFlow provides a foundation for more reliable and natural human-AI collaboration in visual content creation.

Problem

• multi-turn image editing
• error propagation
• exposure bias
• instruction-based image editing
• interactive editing

Innovation

• multi-turn image editing
• flow matching
• reinforcement learning
• reward optimization
• exposure bias mitigation