Small Talk, Big Impact? LLM-based Conversational Agents to Mitigate Passive Fatigue in Conditional Automated Driving

2025-10-29

AI Summary

Passive fatigue in conditional automation poses safety risks by reducing driver alertness and degrading monitoring performance. Method: We propose a Large Language Model (LLM)-driven Context-Aware dialogue Agent (CA) that actively reorients drivers’ attention to the driving environment via natural language interaction. Our approach integrates on-road testing, multimodal data collection (in-vehicle video, drowsiness ratings, user interviews), and user-preference clustering to identify three prototypical driver personas, enabling adaptive dialogue design. Results: Experiments demonstrate that the CA significantly enhances driver alertness and sustains takeover readiness; users consistently affirm its monitoring-assistance value and exhibit high acceptance and usage intent. This work constitutes the first application of an LLM-based dynamic dialogue intervention system for in-vehicle passive fatigue mitigation, providing a scalable methodology and empirical foundation for intelligent human–machine shared-driving interface design.

Technology Category

• Natural Language Processing: Conversational AI/Dialog Systems
• Humans and AI: Intelligent User Interfaces
• Cognitive Modeling & Cognitive Systems: Adaptive Behavior

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Passive fatigue during conditional automated driving can compromise driver readiness and safety. This paper presents findings from a test-track study with 40 participants in a real-world rural automated driving scenario. In this scenario, a Large Language Model (LLM) based conversational agent (CA) was designed to check in with drivers and re-engage them with their surroundings. Drawing on in-car video recordings, sleepiness ratings and interviews, we analysed how drivers interacted with the agent and how these interactions shaped alertness. Users found the CA helpful for supporting vigilance during passive fatigue. Thematic analysis of acceptability further revealed three user preference profiles that implicate future intention to use CAs. Positioning empirically observed profiles within existing CA archetype frameworks highlights the need for adaptive design sensitive to diverse user groups. This work underscores the potential of CAs as proactive Human-Machine Interface (HMI) interventions, demonstrating how natural language can support context-aware interaction during automated driving.

Problem

• Mitigating passive fatigue in automated driving
• Enhancing driver alertness through conversational agents
• Developing adaptive CA designs for diverse users

Innovation

• LLM-based conversational agent for driver engagement
• Proactive HMI intervention using natural language
• Adaptive CA design for diverse user profiles