MT-Web2Code: Benchmarking Coding Agents on Multi-Turn Regional Reconstruction and Localized Modification

2026-08-04

AI Summary

Existing web page generation benchmarks support only single-turn, full-page synthesis, failing to address the practical needs of real-world front-end development, which often involves multi-turn iterative refinement with region-specific reconstruction and localized edits. To bridge this gap, this work introduces the first multimodal coding benchmark tailored for both macro-level region reconstruction and micro-level local editing across 102 tasks spanning 16 vertical domains. We design a scalable backward-contamination trajectory engine to generate deterministic repair paths at low cost and propose a dual-axis evaluation protocol that combines five-dimensional VLM-based scoring with pixel-level code-image alignment to assess target-region fidelity and preservation of unmodified content, respectively. Experiments on 13 state-of-the-art coding agents reveal significant deficiencies in precise region regeneration, content consistency maintenance, and error accumulation control, demonstrating that our benchmark and metrics provide fine-grained feedback essential for training iterative UI agents.

Abstract

Recent advances in Large Vision-Language Models (LVLMs) have demonstrated impressive capabilities in web UI generation. However, existing benchmarks predominantly focus on single-turn full-page generation from scratch, overlooking the iterative workflow of real-world frontend engineering, where developers repeatedly reconstruct missing regions and modify localized elements within existing codebases. To bridge this gap, we introduce MT-Web2Code, the first multimodal coding benchmark for multi-turn Macro-Level Regional Reconstruction and Micro-Level Localized Modification, which contains 102 tasks spanning 16 vertical domains. To construct deterministic repair trajectories without costly turn-level human annotation, we develop a scalable Reverse-Corruption Trajectory Engine that iteratively injects structural and stylistic defects into golden pages. We further propose a dual-axis evaluation protocol that measures target-region fidelity and the preservation of unaffected content, where regional reconstruction is assessed by a 5-dimensional VLM-based rubric and localized modification by deterministic pixel-grounded alignment. Experiments on 13 frontier coding agents reveal that current agents struggle to faithfully reconstruct target regions while preserving unaffected content, lack fine-grained visual-code alignment for localized edits, and suffer from error snowballing over multiple turns. Beyond benchmarking, our deterministic evaluation metrics provide fine-grained feedback signals that may facilitate future research on training iterative UI coding agents. Our evaluation code and data will soon be released.

Problem

• multi-turn
• regional reconstruction
• localized modification
• web UI generation
• coding agents

Innovation

• Multi-turn UI Coding
• Regional Reconstruction
• Localized Modification
• Reverse-Corruption Trajectory
• Dual-axis Evaluation