LiveEvalBench: Toward Open-World Evaluation for Web Generation

2026-08-04

AI Summary

Existing evaluation methods for web page generation treat front-end outputs as static artifacts, struggling to address open-world challenges such as interactivity, implementation diversity, and rapid technological evolution. This work proposes a dynamic, open-world evaluation paradigm centered on multi-agent collaboration, integrating deployment validation, code review, and browser-based interaction testing into an automated framework that holistically assesses generation quality throughout the entire development lifecycle. By combining general-purpose scoring criteria with task-specific guidelines, the framework enables incremental expansion of evaluation roles and dimensions without requiring system redesign. Experimental results demonstrate strong alignment between the framework’s assessments and human expert judgments, offering fine-grained insights into the web generation capabilities of state-of-the-art large language models.

Abstract

Large language models are increasingly capable of synthesizing executable frontend projects, yet existing benchmarks still treat web generation as a static evaluation problem. We argue that frontend artifacts demand a different paradigm: they are interactive rather than static, admit diverse yet equally valid implementations, and evolve faster than rigid pipelines can accommodate. To address these gaps, we present LiveEvalBench, an automated framework that reformulates web-generation evaluation as an agentic, adaptive, and extensible process. LiveEvalBench instantiates evaluation as a collaborative review workflow, in which a Build Engineer, a Code Engineer, and a UI Tester collectively gather evidence across the full lifecycle of a frontend project, from deployment and code inspection to browser-based interaction. To handle implementation diversity, an adaptive protocol couples shared rubrics for cross-model comparability with implementation-grounded criteria tailored to each artifact. The framework further supports incremental integration of new evaluator roles and assessment dimensions without pipeline redesign. Experiments across diverse real-world web-generation scenarios show that LiveEvalBench aligns closely with human expert judgment and provides fine-grained insights into frontier models' web generation capabilities. Code is available at https://github.com/wyysteelhead/LiveEvalBench

Problem

• web generation
• evaluation benchmark
• interactive frontend
• implementation diversity
• open-world evaluation

Innovation

• LiveEvalBench
• web generation evaluation
• agentic evaluation
• adaptive assessment protocol
• interactive frontend artifacts