ReDemon UI: Reactive Synthesis by Demonstration for Web UI

2025-07-14

AI Summary

This work addresses the challenge of efficiently transforming UI prototypes into executable React applications—a task particularly difficult for designers and non-professional developers. We propose an end-to-end React program synthesis method driven by user demonstrations, integrating sketch editing with runtime interaction examples. Our approach leverages event trajectory analysis, reactive dataflow inference, and enumerative program synthesis, augmented by large language models (LLMs) to enhance semantic understanding and code generation. It automatically converts static interface sketches into fully functional React code with correct state management and interactive behavior. Crucially, we pioneer example-driven synthesis in frontend UI generation, enabling unified modeling of simple to moderately complex interfaces while seamlessly embedding into existing prototyping workflows. Experimental evaluation across diverse UI scenarios demonstrates its effectiveness, significantly lowering the frontend development barrier and improving the prototype-to-executable-code translation efficiency.

Technology Category

• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Humans and AI: Intelligent User Interfaces
• Reasoning under Uncertainty: Probabilistic Programming

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

ReDemon UI synthesizes React applications from user demonstrations, enabling designers and non-expert programmers to create UIs that integrate with standard UI prototyping workflows. Users provide a static mockup sketch with event handler holes and demonstrate desired runtime behaviors by interacting with the rendered mockup and editing the sketch. ReDemon UI identifies reactive data and synthesizes a React program with correct state update logic. We utilize enumerative synthesis for simple UIs and LLMs for more complex UIs.

Problem

• Synthesizes React apps from user demonstrations
• Enables non-experts to create UIs via prototyping
• Generates state update logic for reactive data

Innovation

• Synthesizes React apps from user demonstrations
• Uses static mockups with event handler holes
• Combines enumerative synthesis and LLMs