Generative Artificial Intelligence-Guided User Studies: An Application for Air Taxi Services

2024-06-18

Behaviour & Information Technology

AI Summary

This study addresses critical challenges in UX research for emerging urban air mobility (UAM) services—including high safety risks, prohibitive iteration costs, and limited domain expertise—by proposing a generative AI–driven UX research paradigm. Methodologically, it integrates GPT-4–generated scenario scripts with AI-synthesized multimodal stimuli (images and videos) to construct immersive virtual journey experiences, coupled with structural equation modeling (SEM) to examine mediating and moderating mechanisms between user acceptance intention and satisfaction. An empirical evaluation with 72 participants identified key environmental factors significantly enhancing acceptance intention and revealed education level and gender as significant moderators of the intention–satisfaction relationship. To our knowledge, this is the first work to systematically integrate large language model–based content generation, multimodal synthetic stimulus creation, and acceptance factor modeling within UAM UX research. Results demonstrate that GenAI enables effective, efficient early-stage design validation for high-risk, nascent mobility services.

Technology Category

• Humans and AI: User Experience and Usability
• Application Domains: Mobility, Driving & Flight
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• User Modeling, Personalization and Recommendation: Practical large-scale studies of user experience
• Social Networks and Social Media: Generative AI / large language models and their impact on social systems
• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI

Abstract

User studies are crucial for meeting user needs. In user studies, real experimental scenarios and participants are constructed and recruited. However, emerging and unfamiliar studies face limitations, including safety concerns and iterative efficiency. To address these challenges, this study utilises a Generative Artificial Intelligence (GenAI) to create GenAI-generated scenarios for user experience (UX). By recruiting real users to evaluate this experience, we can collect feedback that enables rapid iteration in the early design phase. The air taxi is particularly representative of these challenges and has been chosen as the case study for this research. The key contribution was designing an Air Taxi Journey (ATJ) using Large Language Models (LLMs) and AI image and video generators. Based on the GPT-4-generated scripts, key visuals were created for the air taxi, and the ATJ was evaluated by 72 participants. Furthermore, the LLMs demonstrated the ability to identify and suggest environments that significantly improve participants' willingness toward air taxis. Education level and gender significantly influenced participants' the difference in willingness and their satisfaction with the ATJ. Satisfaction with the ATJ serves as a mediator, significantly influencing participants' willingness to take air taxis. Our study confirms the capability of GenAI to support user studies, providing a feasible approach and valuable insights for designing air taxi UX in the early design phase.

Problem

• Addressing safety and efficiency in user studies for air taxis.
• Using GenAI to create and evaluate air taxi user experiences.
• Analyzing user willingness and satisfaction with AI-generated scenarios.

Innovation

• Generative AI creates air taxi scenarios
• Large Language Models generate UX scripts
• AI-generated visuals evaluate user experience