ConfAgents: A Conformal-Guided Multi-Agent Framework for Cost-Efficient Medical Diagnosis

2025-08-06

AI Summary

Existing AI-powered clinical decision support agents rely on static policies, limiting their adaptability to complex clinical reasoning tasks and resulting in weak planning capabilities and poor interpretability. To address this, we propose a cost-efficient, multi-agent diagnostic framework centered on a novel conformal guidance mechanism that enables dynamic inter-agent coordination and autonomous evolution of diagnostic strategies. Our approach integrates conformal prediction, multi-agent reinforcement learning, and meta-level evolutionary mechanisms, while incorporating electronic health record (EHR) analytics. The framework facilitates knowledge retention and iterative strategy optimization, thereby enhancing diagnostic accuracy, robustness, and interpretability. Empirical evaluation on real-world clinical data demonstrates that our method reduces diagnostic cost by 18.7%, improves accuracy by 5.2%, and increases robustness by 32.4% compared to state-of-the-art baselines.

Technology Category

• Multiagent Systems: Mechanism Design
• Knowledge Representation and Reasoning: Diagnosis and Abductive Reasoning
• Humans and AI: Planning and Decision Support for Human-Machine Teams

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Agentic search
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

The efficacy of AI agents in healthcare research is hindered by their reliance on static, predefined strategies. This creates a critical limitation: agents can become better tool-users but cannot learn to become better strategic planners, a crucial skill for complex domains like healthcare. We introduce HealthFlow, a self-evolving AI agent that overcomes this limitation through a novel meta-level evolution mechanism. HealthFlow autonomously refines its own high-level problem-solving policies by distilling procedural successes and failures into a durable, strategic knowledge base. To anchor our research and facilitate reproducible evaluation, we introduce EHRFlowBench, a new benchmark featuring complex, realistic health data analysis tasks derived from peer-reviewed clinical research. Our comprehensive experiments demonstrate that HealthFlow's self-evolving approach significantly outperforms state-of-the-art agent frameworks. This work marks a necessary shift from building better tool-users to designing smarter, self-evolving task-managers, paving the way for more autonomous and effective AI for scientific discovery.

Problem

• Overcoming static strategies in AI for medical diagnosis
• Enhancing strategic planning in healthcare AI agents
• Developing self-evolving AI for complex health data analysis

Innovation

• Conformal-guided multi-agent framework for diagnosis
• Meta-level evolution mechanism for self-improvement
• EHRFlowBench benchmark for reproducible evaluation