DesignCoder: Hierarchy-Aware and Self-Correcting UI Code Generation with Large Language Models

2025-06-16

AI Summary

To address the core challenge in multimodal large language models (MLLMs)—generating frontend code that simultaneously ensures visual fidelity and functional completeness—this paper proposes a hierarchical-aware and self-correcting end-to-end code generation framework. We innovatively model UI nesting hierarchies via UI Grouping Chains and adopt a divide-and-conquer code generation strategy. Furthermore, we design a self-correction mechanism jointly evaluating rendered-image fidelity (MSE, SSIM, CLIP) and structural similarity (TreeBLEU, Container Match, Tree Edit Distance). Experiments on React Native demonstrate significant improvements: visual similarity increases by 9.52%–37.63%, and structural similarity by 24.67%–30.19%, substantially outperforming state-of-the-art methods. Developer evaluations confirm high usability, code readability, and industrial-grade maintainability.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Computer Vision: Multi-modal Vision

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG

Abstract

Multimodal large language models (MLLMs) have streamlined front-end interface development by automating code generation. However, these models also introduce challenges in ensuring code quality. Existing approaches struggle to maintain both visual consistency and functional completeness in the generated components. Moreover, they lack mechanisms to assess the fidelity and correctness of the rendered pages. To address these issues, we propose DesignCoder, a novel hierarchical-aware and self-correcting automated code generation framework. Specifically, we introduce UI Grouping Chains, which enhance MLLMs' capability to understand and predict complex nested UI hierarchies. Subsequently, DesignCoder employs a hierarchical divide-and-conquer approach to generate front-end code. Finally, we incorporate a self-correction mechanism to improve the model's ability to identify and rectify errors in the generated code. Extensive evaluations on a dataset of UI mockups collected from both open-source communities and industry projects demonstrate that DesignCoder outperforms state-of-the-art baselines in React Native, a widely adopted UI framework. Our method achieves a 37.63%, 9.52%, 12.82% performance increase in visual similarity metrics (MSE, CLIP, SSIM) and significantly improves code structure similarity in terms of TreeBLEU, Container Match, and Tree Edit Distance by 30.19%, 29.31%, 24.67%. Furthermore, we conducted a user study with professional developers to assess the quality and practicality of the generated code. Results indicate that DesignCoder aligns with industry best practices, demonstrating high usability, readability, and maintainability. Our approach provides an efficient and practical solution for agile front-end development, enabling development teams to focus more on core functionality and product innovation.

Problem

• Ensuring visual consistency and functional completeness in UI code generation
• Assessing fidelity and correctness of rendered UI pages
• Improving error identification and correction in generated code

Innovation

• Hierarchy-aware UI Grouping Chains for nested structures
• Hierarchical divide-and-conquer code generation approach
• Self-correction mechanism for error identification and rectification