Clinician-Grounded Quality Assurance for AI-Assisted Psychiatric Intake

2026-09-17

AI Summary

本文提出了一种基于临床医生的评估平台，通过模拟患者和记忆增强技术来评价AI辅助精神科接诊系统，以确保其符合临床标准。

Abstract

Before patients can use AI-assisted psychiatric intake systems, health systems need practical ways to routinely evaluate these tools against their clinical standards for quality assurance. Because clinicians may use different intake styles, evaluation for this task must (1) support comparison across interviewing approaches, (2) minimize clinician burden, and (3) measure clinically relevant performance for health systems deploying these technologies. We present a clinician-grounded evaluation platform built around a memory-augmented patient simulator for open-ended AI interviewing, InterviewPlayground. We created interactive patients using InterviewPlayground with our expert-authored vignettes, constructed a simulated intake platform for the interviews, and designed evaluation modalities relevant to intake. In a pilot of 6 clinicians in a 25-minute assessment compared to a GPT-based LLM intake interviewer, the LLM recovered more of the clinically relevant items embedded in the patient vignettes (88.0% vs. 38.9%), but made more clinical inferences not based on the interview (56.8% vs. 27.8%), and characterized identified safety concerns less often (33.3% vs. 66.7%), setting the stage for deployed quality assurance for this task.

Problem

• Quality Assurance
• AI-Assisted Psychiatric Intake
• Clinical Standards
• Clinician Burden

Innovation

• Clinician-Grounded Evaluation
• Memory-Augmented Patient Simulator
• InterviewPlayground
• Quality Assurance
• AI-Assisted Psychiatric Intake