clinical relevance assessment

Current medical AI evaluation benchmarks predominantly emphasize knowledge acquisition, failing to adequately capture model reliability, safety, and clinical utility in real-world settings. To address this gap, this work proposes the first systematic evaluation framework aligned with clinical workflows, encompassing end-to-end tasks such as clinical documentation, decision support, and administrative processes. The framework integrates authentic multimodal clinical data and introduces task-specific metrics to comprehensively assess generative models, multimodal systems, and AI agents. Empirical results reveal a substantial performance gap between state-of-the-art models on real-world tasks and their scores on medical knowledge exams—scoring 0.74–0.85 in documentation, 0.61–0.76 in clinical decision-making, and 0.53–0.63 in administrative tasks—highlighting the limitations of existing evaluation paradigms and underscoring the critical role of this framework in advancing the clinical deployment of medical AI.

Fairness evaluation of clinical predictive AI has long suffered from conceptual ambiguity, strong threshold dependence, lack of clinical validation, and insufficient quantification of intersectionality and uncertainty. To address these gaps, we conducted a scoping review (2014–2024) across five major academic databases, systematically identifying and analyzing 62 fairness metrics from 41 studies. We propose the first three-dimensional taxonomy—structured along *performance dependence*, *output level*, and *benchmark type*—revealing severe fragmentation: only 18 metrics are healthcare-specific, and merely one is explicitly oriented toward clinical utility. Our analysis identifies three critical deficiencies: (1) absence of clinically meaningful fairness metrics; (2) inadequate uncertainty quantification; and (3) limited intersectional modeling and real-world applicability. This work establishes a theoretical foundation and practical roadmap for transitioning fairness assessment from methodological abstraction to clinical implementation.

This work proposes the first risk-sensitive hallucination evaluation framework for medical question-answering models, addressing a critical gap in existing assessments that focus narrowly on factual accuracy while overlooking variations in clinical risk. By identifying risk-laden language—such as treatment directives, contraindications, urgent warnings, and high-risk medications—and integrating relevance-based metrics, the framework quantifies the potential harm arising from user adherence to hallucinated responses. It introduces a clinical risk dimension to distinguish between low- and high-risk hallucinations, revealing safety disparities invisible to conventional evaluation metrics. Experiments on three leading instruction-tuned large language models demonstrate that models exhibiting similar surface-level behaviors can possess markedly different risk profiles, underscoring the necessity and effectiveness of this approach for comprehensive safety assessment in medical AI systems.

Current health AI evaluation benchmarks lack standardized descriptions of user queries, limiting their ability to accurately reflect model applicability in real-world clinical settings. This study systematically identifies this “validity gap” and proposes adapting clinical trial reporting standards to create structured query profiles. Leveraging large language models, we automatically annotated 18,707 health-related queries from six public benchmarks using a 16-dimensional taxonomy capturing clinical context, topic, and intent. Our analysis reveals significant structural biases: existing benchmarks severely underrepresent complex diagnostic information such as laboratory tests, imaging, and raw clinical notes; safety-critical scenarios (e.g., self-harm) constitute less than 0.7% of queries; and coverage of pediatric, geriatric, and chronic disease populations is markedly insufficient—highlighting a substantial misalignment between current evaluation frameworks and actual clinical needs.

Medical foundation models may generate “hallucinations”—factual, logical, or evidence-inconsistent errors—that jeopardize clinical decision-making and patient safety. To address this, we first propose a multidimensional taxonomy of medical hallucinations and establish a real-world, clinician-annotated benchmark dataset derived from authentic clinical cases; we further validate its clinical impact via an international physician survey. Methodologically, we integrate expert annotation, empirical behavioral surveys, and large language model (LLM) evaluation to systematically assess the efficacy of chain-of-thought (CoT) reasoning and retrieval-augmented generation (RAG) in mitigating hallucinations. Results show both techniques significantly reduce hallucination rates, yet residual hallucinations remain clinically hazardous. Building on these findings, we introduce a patient-safety-centered AI governance and ethics framework, offering theoretical foundations and actionable pathways for responsible deployment of medical AI. (149 words)

Clinical deep learning models face significant challenges in high-stakes settings due to the lack of reliable and generalizable interpretability, hindering their validation and deployment. This work presents the first systematic evaluation of multiple interpretability methods—including attention mechanisms, KernelSHAP, and LIME—across diverse deep temporal architectures and multitask clinical prediction scenarios. The study demonstrates that, when appropriately applied, attention mechanisms offer both computational efficiency and faithfulness to the underlying model, whereas KernelSHAP and LIME suffer from intractable computational demands or insufficient reliability in temporal tasks. Built upon the PyHealth framework, the project establishes the first reproducible and extensible benchmark for clinical interpretability and provides practical guidelines to inform future research, with all code publicly released to foster community advancement.

Current large language models lack structured evaluation of core therapeutic principles in mental health conversations, compromising clinical appropriateness. To address this gap, this work introduces FAITH-M—the first expert-annotated benchmark grounded in established therapeutic principles—and CARE, a multi-stage evaluation framework that enables precise assessment of AI therapist responses through fine-grained ordinal scoring, context-aware analysis, contrastive example retrieval, and chain-of-thought knowledge distillation. Using Qwen3 as the backbone model, CARE achieves an F1 score of 63.34, representing a 64.26% improvement over baseline methods, and demonstrates strong robustness across diverse datasets and expert evaluations.

Current evaluations of medical AI systems lack transparency in reasoning processes, fine-grained assessment of atomic clinical skills, and robust hallucination detection. This work proposes the first dynamic, process-oriented multimodal evaluation framework for clinical AI, integrating language, vision-language, and agent-based systems. It introduces a dual-dimensional assessment combining clinical cognitive responses and medical atomic skills, a five-node dynamic reasoning audit trail, three switchable information-flow perturbation strategies, and a cross-stage hallucination propagation tracking mechanism to jointly analyze model reasoning stability and hallucination trajectories. Experimental results reveal that high overall performance does not necessarily imply robust reasoning: information-flow stress significantly impairs contradiction detection, diagnostic updating, and self-correction of hallucinations, suggesting that seemingly stable final outputs may merely reflect superficial consistency rather than genuine reasoning reliability.

Current static, single-turn evaluations inadequately capture the comprehensive capabilities of large language models in dynamic clinical settings—particularly their ability to gather information, formulate treatment plans, and manage long-term care. To address this gap, this work proposes MedSP1000, the first interactive evaluation benchmark built upon standardized patient teaching cases. It leverages a closed-loop simulation system comprising structured scenario scripts, an environment controller, and patient agents to enable end-to-end, automatically scored assessments, augmented by expert-reviewed, trajectory-level scoring criteria. Experiments reveal that even the strongest general-purpose model (GPT-5.5) achieves only 60.4% of expert-rated performance, while domain-specific medical models perform substantially worse (40.0%). Moreover, increased reasoning compute yields no significant gains, exposing critical clinical failure modes invisible to static benchmarks and underscoring that current models remain unsafe for real-world clinical deployment.

Current large language models (LLMs) lack systematic evaluation of interpretive reliability in psychiatric clinical risk assessment and are susceptible to interference from non-clinical information and prompt design. This study introduces the first LLM reliability auditing framework tailored to psychiatry, systematically evaluating the stability of hospitalization risk scores across four leading models—Gemini, LLaMA, Claude, and GPT—using synthetically generated patient profiles that incorporate both clinical and non-clinical features, alongside four prompting strategies: neutral, logical, humanistic influence, and clinical judgment. Results demonstrate that non-clinical information significantly increases both the mean risk scores and output variability across all models, revealing a high sensitivity to contextual noise and underscoring substantial reliability concerns for clinical deployment.