Assessing the Feasibility of a Video-Based Conversational Chatbot Survey for Measuring Perceived Cycling Safety: A Pilot Study in New York City

2026-04-07

AI Summary

This study addresses the limitations of traditional questionnaires, which rely on retrospective recall and struggle to accurately capture cyclists’ real-time perceptions of environmental safety. To overcome this, the authors propose an innovative approach integrating immersive street-view videos with a modular large language model (LLM)-driven conversational chatbot that elicits users’ immediate safety perceptions and their attributions through structured human–AI interaction. This method uniquely combines contextual video stimuli with a controllable and interpretable LLM dialogue system, enhancing both ecological validity and user experience. The system incorporates prompt engineering, state management, and rule-based constraints, and leverages KeyBERT, K-means clustering, and regression analysis to effectively extract built environment attributes and safety-related factors. User evaluations yielded a usability score of 5.00/7 and a chatbot-specific score of 3.47/5, demonstrating the feasibility and novel contribution of the proposed framework.

Abstract

Bicycle safety is important for bikeability and transportation efficiency. However, conventional surveys often fall short in capturing how people actually perceive cycling environments because they rely heavily on respondents' recall rather than in-the-moment experience. By leveraging large language models (LLMs), this study proposes a new method of combining video-based surveys with a conversational AI chatbot to collect human perceptions of cycling safety and the reasons behind these perceptions. The paper developed the AI chatbot using a modular LLM architecture, integrating prompt engineering, state management, and rule-based control to support the structure of human-AI interaction. This paper evaluates the feasibility of the proposed video-based conversational chatbot using complete responses from sixteen participants to the pilot survey across nine street segments in New York City. The method feasibility was assessed using a seven-point scale rating for user experience (i.e., ease of use, supportiveness, efficiency) and a five-point scale for chatbot usability (i.e., personality, roboticness, friendliness), yielding positive results with mean scores of 5.00 out of 7 (standard deviation = 1.6) and 3.47 out of 5 (standard deviation = 0.43), respectively. The data feasibility was assessed using multiple techniques: (1) Natural language processing (NLP), such as KeyBERT, for overall safety and feature analysis to extract built-environment attributes; (2) K-means clustering for semantic analysis to identify reasons and suggestions; and (3) regression to estimate the effects of built-environment and demographic variables on perceived safety outcomes. The results show the potential of AI chatbots as a novel approach to collecting data on human perception, behavior, and future visions for transport planning.

Problem

• perceived cycling safety
• video-based survey
• conversational chatbot
• human perception
• transportation planning

Innovation

• video-based survey
• conversational AI chatbot
• large language models
• perceived cycling safety
• modular LLM architecture