WebUIProof: Benchmarking WebUI Code Generators with UI-Agent Execution Harness

2026-10-01

AI Summary

This study addresses the challenge that static evaluation cannot verify interaction correctness in WebUI code generation. To this end, it proposes an execution-oriented benchmark alongside a novel UI agent execution framework. This framework establishes structured specifications encompassing both general and 3D interfaces, employing a headless browser to drive a plan-act-observe loop for dynamically validating DOM interactions and assertions. Furthermore, it integrates reinforcement learning reward mechanisms to optimize model performance. The research reveals high failure rates among large language models in complex interactive tasks and demonstrates that reinforcement learning training grounded in execution feedback significantly improves functional completion rates.

Abstract

Evaluating WebUI code generation at scale is difficult: outputs may compile and look plausible yet fail under user interaction, and prior benchmarks largely rely on free-form prompts with static checks (build success, screenshots) that miss functional correctness. We introduce WebUIProof, an execution-oriented benchmark that provides structured specifications and dense, executable interaction tests for WebUI generation across two task families: general WebUIs (e.g., dashboards, game, interactive tools) and 3D interactive simulation (e.g., particle/galaxy systems, physics dynamics). WebUIProof includes a UI-agent harness that runs executable interaction tests in a headless browser using an iterative plan--act--observe loop: it locates DOM elements, performs actions, observes resulting UI/DOM changes, and checks the specified assertions. We evaluate across eight commercial LLMs and observe frequent failures on interaction-based requirements even when pages render successfully, especially on 3D simulation interfaces. Finally, we show the UI-agent harness can provide outcome-level training signals. Training compact models (e.g., Qwen2.5 14B and MIMO 7B) with RL rewards derived from executable interaction tests improves functional completion while reducing build failures.

Problem

• WebUI code generation
• functional correctness
• benchmark evaluation
• interaction testing

Innovation

• WebUI code generation
• execution-oriented benchmark
• UI-agent harness
• reinforcement learning
• executable interaction tests