SimClinician: A Multimodal Simulation Testbed for Reliable Psychologist AI Collaboration in Mental Health Diagnosis

2025-11-28

AI Summary

Existing AI systems for mental health diagnosis prioritize benchmark accuracy while neglecting clinicians’ actual adoption and modification of AI recommendations in real-world practice. Method: Addressing the continuous, multimodal nature of diagnostic cues—such as prosody, pauses, lexical choice, gaze, and micro-expressions—we develop the first interactive, collaborative diagnostic simulation platform supporting real-time reasoning inspection, multimodal evidence traceability, and dynamic feedback. Contributions/Results: We introduce (1) de-identified multimodal virtual patient avatars; (2) a diagnostic decision-layer mapping mechanism that explicitly grounds AI outputs in audio, textual, and gaze–facial dynamics evidence; and (3) the first quantitative analysis of UI design impact on recommendation acceptance and false escalation rates. Across 480,000 simulations, adding a confirmation step increased clinician acceptance of AI suggestions by 23%, reduced false escalation to ≤9%, and significantly improved interaction fluency.

Technology Category

• Humans and AI: Intelligent User Interfaces
• Knowledge Representation and Reasoning: Diagnosis and Abductive Reasoning
• Cognitive Modeling & Cognitive Systems: Simulating Human Behavior

Application Category

• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Search and Retrieval-Augmented AI: Assisted, interactive, and conversational search
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

AI based mental health diagnosis is often judged by benchmark accuracy, yet in practice its value depends on how psychologists respond whether they accept, adjust, or reject AI suggestions. Mental health makes this especially challenging: decisions are continuous and shaped by cues in tone, pauses, word choice, and nonverbal behaviors of patients. Current research rarely examines how AI diagnosis interface design influences these choices, leaving little basis for reliable testing before live studies. We present SimClinician, an interactive simulation platform, to transform patient data into psychologist AI collaborative diagnosis. Contributions include: (1) a dashboard integrating audio, text, and gaze-expression patterns; (2) an avatar module rendering de-identified dynamics for analysis; (3) a decision layer that maps AI outputs to multimodal evidence, letting psychologists review AI reasoning, and enter a diagnosis. Tested on the E-DAIC corpus (276 clinical interviews, expanded to 480,000 simulations), SimClinician shows that a confirmation step raises acceptance by 23%, keeping escalations below 9%, and maintaining smooth interaction flow.

Problem

• Evaluates AI's impact on psychologist decision-making in mental health diagnosis.
• Addresses lack of testing for AI interface design in clinical collaboration.
• Measures how multimodal evidence influences acceptance and adjustment of AI suggestions.

Innovation

• Interactive simulation platform for psychologist-AI collaboration
• Dashboard integrates audio, text, and gaze-expression patterns
• Decision layer maps AI outputs to multimodal evidence