user interface design

Large-scale wall-mounted displays pose significant challenges for existing UI design tools, particularly due to inadequate support for 1:1 scale design and poor adaptability to physical display dimensions. Method: We conducted two user studies and technical evaluations comparing three input modalities—touchscreen, trackpad–keyboard, and tablet—in two wall-display environments. Contribution/Results: Results demonstrate that 1:1 scale design substantially improves design accuracy and user acceptance; tablet-based interaction achieves superior comfort and task efficiency; and hybrid interaction shows promising applicability. Based on empirical findings, we propose 12 domain-specific design guidelines for wall-display environments, advancing a novel design paradigm centered on “multimodal interaction + environment-aware adaptation.” We further identify critical limitations in current UI tooling—including inflexible layout logic and insufficient native support for large-display workflows—and recommend targeted enhancements to address them.

Empirical usability research on mobile applications remains scarce, hindering systematic understanding of critical usability challenges. Method: We conducted semi-structured interviews with 12 industry experts, followed by thematic coding and consensus analysis to identify core usability issues and emerging research directions. Contribution/Results: We systematically delineate five fundamental usability dimensions—information architecture, interface design, performance, interaction patterns, and aesthetics—and, for the first time, articulate five frontier research avenues: AI-driven applications, AR/VR integration, multimodal interaction, personalized mobile ecosystems, and accessibility. This work bridges industrial insights with academic agendas, yielding a comprehensive usability problem landscape and an actionable research roadmap. It advances human–computer interaction (HCI) theory while providing bidirectional support for industrial design practice.

This work addresses the challenge that existing generative user interfaces (GenUIs) struggle to support effective discovery of diverse customization options while maintaining expressive power. To overcome this, the paper proposes a progressive GenUI approach that introduces a structured intermediate UI layer into the AI generation pipeline, incrementally exposing customization capabilities along specific dimensions and enabling users to backtrack through the generation process to explore and refine designs. By decomposing the interface generation into traceable, intermediate representations—a novel contribution—the method effectively balances system expressiveness with the discoverability of customization features. A prototype system implementing this approach integrates generative AI, interaction design, and hierarchical customization mechanisms. Evaluations across three case studies demonstrate its ability to significantly enhance users’ exploration of complex customization spaces while preserving visual simplicity, offering a new paradigm for malleable software design.

This study addresses the challenge of enhancing UI design efficiency, diversity, and creative quality through AI augmentation. We propose a human-AI collaborative “AI-inspired” design paradigm, systematically integrating large language models (LLMs), vision-language models (VLMs), and diffusion models (DMs) fine-tuned for UI generation. Our method comprises three technical pathways: (1) LLM-driven UI specification, generation, and iterative refinement; (2) VLM-enabled cross-modal semantic retrieval over application screenshots; and (3) high-fidelity UI image synthesis via domain-adapted DMs. To our knowledge, this is the first work to achieve organic, end-to-end synergy among these three state-of-the-art AI model classes in the UI design workflow—while preserving human designers’ creative agency. Empirical evaluation demonstrates significant gains in inspiration stimulation, iteration acceleration, and solution diversification. We deliver a production-ready workflow, rigorously delineate technical boundaries, and identify critical ethical challenges—including attribution, bias, and design autonomy.

Existing visualization research predominantly focuses on *how to use* interactive features, neglecting the critical question of *how to construct* them. Method: We propose the first three-layer decoupled interaction authoring task model—intent–technique–component—derived from empirical coding and abstraction of 592 interaction units across 47 real-world applications. Contribution/Results: This model provides descriptive, evaluative, and generative capabilities, enabling the first unified formalization of interaction authoring intent, technical implementation, and component instantiation. It yields a reusable, theory-grounded classification framework that supports critical evaluation of existing visualization tools and informs the design and validation of next-generation low-code interaction authoring systems.

This work addresses the inefficiency and lack of guidance faced by front-end developers when manually selecting plausible and natural attribute values for instantiating reusable UI components within a vast design space. To tackle this challenge, the paper introduces the concept of “discriminative variants,” which uniquely integrates symbolic reasoning with large language models (LLMs). Symbolic reasoning identifies visually salient attributes, while the LLM leverages real-world knowledge to generate component instances that balance fidelity to exemplars with meaningful differentiation. This approach shifts the paradigm from ad hoc manual configuration to structured exploration of the design space. A user study (n=12) demonstrates that the generated variants effectively aid developers in comprehending the design space, significantly improving both instantiation efficiency and user experience, while maintaining strong domain relevance.

Directly scaling desktop visualizations to mobile devices often results in unreadable text, loss of information, and broken interactions. This work proposes the first multi-granularity adaptive framework that spans topological structure, reference frames, and visual elements, coupled with a large language model–based multi-agent system to automate the transformation pipeline—from parsing and strategy prediction to mobile-ready visualization generation. User studies (N=12) and case evaluations demonstrate that the approach efficiently produces mobile visualizations with high readability and intuitive interactivity, significantly enhancing user experience.

This work addresses the challenges faced by multimodal large language models (MLLMs) in automatically annotating mobile user interfaces, which are often densely populated, hierarchically nested, and visually ambiguous, leading to limited annotation accuracy and high sensitivity to prompt design and task formulation. To mitigate these issues, the authors propose a context-aware, staged annotation workflow that decomposes the overall task into multiple coordinated phases through structured prompting, schema-constrained JSON output, and element-specific instructions. Experimental results demonstrate that this approach significantly enhances the reliability of UI understanding: on the MUIAnno dataset, a two-stage pipeline achieves the highest precision, while deeper task decomposition improves recall at the cost of increased false positives, revealing a joint influence of decomposition depth and element category grouping on annotation quality.

This study addresses the prevalent issue of “design theater” in generative UI tools—where plausible-sounding design rationales are provided but not actually implemented. The work introduces and quantifies this problem for the first time, establishing a benchmark comprising 24 tasks and three evaluation dimensions: structure, style, and functionality. A systematic assessment of five state-of-the-art tools reveals that over 25% of stated design rationales remain unimplemented on average, with a functional requirement failure rate of 34%. Furthermore, these tools recognize only about 54% of relevant UX principles, and most achieve functional implementation rates below 6%. This research presents the first evaluation framework specifically targeting design-implementation consistency in generative UI systems, offering a new benchmark for trustworthy human-AI interface generation.