TeleGen: Improving LLM-Based Web Application Generation via Runtime Telemetry

2026-10-04

AI Summary

This study addresses the challenge of diagnosing interaction failures in web applications generated by large language models (LLMs), where existing feedback mechanisms lack sufficient runtime details. To this end, we propose TeleGen, a framework that introduces an observability-enhancement mechanism. By instrumenting the application to collect runtime telemetry data and employing a log compression algorithm to distill these data into concise briefs, TeleGen provides precise diagnostic signals for LLM-based code repair, effectively resolving failures caused by latent execution paths. Experimental evaluations on the WebGen-Bench and Web-Bench benchmarks demonstrate that TeleGen significantly improves task success rates and Pass@2 metrics while substantially reducing repair costs.

Abstract

Large language models can generate runnable web applications from natural-language requirements, but many generated applications still fail interactive tasks. Existing generate-execute-repair pipelines execute the generated application and use task outcomes or error messages to guide code revision. However, this feedback often misses the runtime behavior between a browser action and the final task outcome, making interaction-level failures difficult to diagnose. Therefore, we propose TeleGen, an observability-enhanced framework for LLM-based web application generation. TeleGen instruments generated applications, collects runtime telemetry during task execution, and compresses raw telemetry logs into concise briefs for repair. We evaluate TeleGen on WebGen-Bench and Web-Bench. On WebGen-Bench, TeleGen improves task success from 67.7% with repair without telemetry to 76.2%, an increase of 8.5 percentage points. On Web-Bench, it improves cumulative Pass@2 from 21.7% to 29.8%. Ablation results show that runtime telemetry provides a useful diagnostic signal, while telemetry briefs make this signal more effective and less costly to use. Further analysis shows that telemetry is especially helpful for failures involving hidden execution paths, such as navigation, form workflows, and frontend-backend coordination.

Problem

• LLM-based web application generation
• interactive task failure
• runtime telemetry
• generate-execute-repair pipeline
• fault diagnosis

Innovation

• Runtime Telemetry
• Web Application Generation
• Generate-Execute-Repair Pipeline
• Telemetry Compression
• Large Language Models