LLMs Can Simulate Standardized Patients via Agent Coevolution

2024-12-16

arXiv.org

AI Summary

Current standardized patient (SP) training relies heavily on expert knowledge and manual annotation, resulting in high costs and poor generalizability. To address this, we propose EvoPatient—a novel framework featuring a dual-agent co-evolutionary mechanism: a patient agent and a physician agent jointly refine standardized clinical interview responses through unsupervised, multi-round dialogues, without human feedback or retrieval augmentation. Our method integrates large language model (LLM)-based multi-agent systems, requirement-aligned evaluation, and human preference optimization. Experiments across diverse clinical cases demonstrate over 10% improvement in requirement alignment and statistically significant gains in human preference scores. Moreover, evolving 200 SP instances requires only 10 hours—achieving state-of-the-art resource efficiency and strong cross-case generalization. To our knowledge, EvoPatient is the first framework enabling end-to-end, fully unsupervised emergence of SP capabilities, establishing a new paradigm for intelligent clinical training.

Technology Category

• Cognitive Modeling & Cognitive Systems: Agent Architectures
• Multiagent Systems: Adversarial Agents
• Machine Learning: Evolutionary Learning

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Agentic search

Abstract

Training medical personnel using standardized patients (SPs) remains a complex challenge, requiring extensive domain expertise and role-specific practice. Most research on Large Language Model (LLM)-based simulated patients focuses on improving data retrieval accuracy or adjusting prompts through human feedback. However, this focus has overlooked the critical need for patient agents to learn a standardized presentation pattern that transforms data into human-like patient responses through unsupervised simulations. To address this gap, we propose EvoPatient, a novel simulated patient framework in which a patient agent and doctor agents simulate the diagnostic process through multi-turn dialogues, simultaneously gathering experience to improve the quality of both questions and answers, ultimately enabling human doctor training. Extensive experiments on various cases demonstrate that, by providing only overall SP requirements, our framework improves over existing reasoning methods by more than 10% in requirement alignment and better human preference, while achieving an optimal balance of resource consumption after evolving over 200 cases for 10 hours, with excellent generalizability. The code will be available at https://github.com/ZJUMAI/EvoPatient.

Problem

• Training medical personnel using standardized patients requires complex role-specific expertise
• Existing LLM-based SPs lack standardized presentation patterns for human-like responses
• Current methods overlook unsupervised simulation for improving diagnostic dialogue quality

Innovation

• Agent coevolution for patient simulation
• Unsupervised learning for human-like responses
• Multi-turn dialogue for diagnostic training