WebGrader: Training LLMs for Web Development with Self-Evolving Programmatic Grader

2026-08-06

AI Summary

This work addresses the limitations of existing reinforcement learning approaches in web generation tasks, which are hindered by poor-quality reward signals—either due to the high cost of handcrafted scripts or the inability of vision-language models and GUI agents to accurately identify critical states. To overcome this, the authors propose WebGrader, a self-evolving procedural grader that automatically derives interaction workflows from natural language specifications, generates executable Flow Contracts, and validates target state transitions within a real browser using multimodal evidence (visual, DOM, responses, and persistent states). Its core innovation lies in decoupling test planning, action anchoring, evidence collection, and semantic judgment, augmented by a residual-driven offline loop that discovers reusable verification skills. Experiments show that an 8B policy model trained on WebGen-Bench achieves a functional success rate of 52.01%, outperforming baselines by 7.88 percentage points, and attains a Full Score of 44.953 on WG-core-250, surpassing o4-mini, DeepSeek-v4-flash, and Qwen3-Coder-480B.

Abstract

Large language models increasingly generate complete websites from natural-language descriptions, and reinforcement learning has become a central approach to closing their remaining functional gap. This training regime is bottlenecked by reward design. Hand-authored browser scripts are executable yet costly to write for open-ended requirements, while VLM and GUI-agent graders scale but may issue verdicts before observing the decisive state. We propose WebGrader, a self-evolving programmatic grader that autonomously derives the required interaction flows from each website request, represents each flow as an executable Flow Contract, and uses its execution outcome as an RL reward. WebGrader materializes the generated project in a live browser, grounds target actions against the source code and live DOM, and collects visual, DOM, response, and persistent-state evidence along the same browser trajectory. A residual-driven offline loop then discovers reusable verifier skills, screens them on disjoint validation pages, and freezes the promoted skill graph before policy training. By separating test planning, action grounding, evidence collection, and semantic judgment, WebGrader issues a Pass verdict only after observing the requested transition. On WebGen-Bench, WebGrader trains an 8B policy to a 52.01% functional success rate, outperforming a matched appearance-plus-script reward by 7.88 points and surpassing o4-mini and DeepSeek-v4-flash. On WG-core-250, the policy reaches a Full Score of 44.953 and surpasses Qwen3-Coder-480B.

Problem

• reward design
• web development
• large language models
• functional evaluation
• reinforcement learning

Innovation

• WebGrader
• self-evolving grader
• Flow Contract
• reinforcement learning for web development
• programmatic reward