explainable ai

This paper addresses the challenge of AI’s “black-box” nature impeding expert collaboration in scientific discovery during the AI era. We propose “Interpretability-Guided Exploration” (IGE), a novel paradigm that redefines interpretability as a cognitive interface between scientists and AI—specifically, as *scientifically oriented explainable AI*—to jointly support divergent hypothesis generation and convergent knowledge validation. Methodologically, IGE integrates intrinsically interpretable models, attribution techniques (e.g., SHAP, LIME), generative AI–augmented reasoning, and cross-disciplinary workflow modeling. We establish a formal theoretical framework and collaborative pathway for IGE, providing an actionable methodology for AI-driven hypothesis formulation, experimental design, and iterative knowledge refinement. The approach significantly enhances AI’s substantive contribution to scientific exploration in the pre-AGI stage, bridging the gap between technical interpretability and domain-specific scientific reasoning.

Existing XAI tools for high-stakes medical diagnosis rely on users’ secondary interpretation, exacerbating the comprehension gap and undermining trust. This paper addresses cardiac auscultation diagnosis by proposing DiagramNet—a novel ante-hoc interpretable model that uniquely integrates domain ontology–driven schematic representation with abductive reasoning to automatically generate clinically intelligible murmur waveform explanations. DiagramNet jointly generates diagnostic hypotheses and produces visualizations explicitly aligned with clinical knowledge, substantially narrowing the interpretability gap between model outputs and expert understanding. Experimental results demonstrate that DiagramNet outperforms baseline models in diagnostic accuracy. A user study with practicing physicians confirms significantly higher trust in its schematic explanations compared to heatmap-based alternatives; moreover, explanation faithfulness and diagnostic credibility co-improve. This work advances trustworthy, clinically grounded XAI by bridging symbolic domain knowledge with data-driven inference in an end-to-end, human-centered framework.

This study addresses the lack of a domain-agnostic, human-centered explainable artificial intelligence (XAI) framework by investigating user preferences across healthcare, retail, and energy domains. Through expert interviews and multi-stakeholder structured surveys, it empirically identifies “interpretability over accuracy” as a cross-domain preference and establishes feature importance and counterfactual explanations as the two foundational pillars of a universal XAI framework. Method: The approach integrates qualitative transcription analysis, questionnaire-driven requirement modeling, and genetic programming (GP) to construct inherently interpretable models. Contribution/Results: We propose the first empirically validated, unified XAI framework spanning multiple domains; release an open-source, standardized XAI questionnaire toolkit; demonstrate its feasibility across three core machine learning tasks—prediction, diagnosis, and prescription—and advance the XAI paradigm from technology-centric design toward human consensus–driven development.

In high-stakes human-AI collaborative medical decision-making, clinicians often over-rely on AI systems, while existing AI assistance suffers from poor interpretability and insufficient task adaptability. Method: This paper proposes the “Learning-to-Guide” (LTG) framework—a human-centered paradigm that preserves ultimate clinical authority and auditability by leveraging vision-language models (VLMs) to generate interpretable, task-specific textual decision guidance—not autonomous decisions. We introduce SLOG (Supervised Learning of Guidance), a lightweight method that efficiently adapts arbitrary VLMs into high-quality guidance generators using minimal human feedback, integrating instruction tuning, human-feedback-driven prompt optimization, and interpretable text generation. Contribution/Results: Evaluated on real-world diagnostic tasks, LTG significantly improves physician diagnostic accuracy and inter-rater consistency while reducing cognitive load—all without compromising human oversight or system controllability.

This study addresses the interpretability and biological plausibility of neuro-symbolic AI for mechanism-of-action (MoA) deconvolution in drug discovery, specifically tackling reasoning shortcuts and prediction biases induced by degree bias in knowledge graph models. To this end, we propose (i) the first dedicated evaluation paradigm for MoA deconvolution; (ii) MoA-net, a structured prior knowledge graph encoding domain-specific pharmacological relationships; and (iii) MARS, a neuro-symbolic system that jointly learns logical rules and graph embeddings for end-to-end, biologically grounded inference. We explicitly mitigate degree bias through diagnostic analysis and targeted regularization. Experiments demonstrate that MARS achieves state-of-the-art performance on MoA prediction while generating inference paths that align closely with established pharmacological mechanisms. This significantly enhances model transparency, scientific credibility, and practical utility in biomedical discovery.

Current machine learning evaluation practices predominantly rely on surface-level performance metrics, often neglecting the internal mechanisms of models. This work proposes trustworthy interpretability as a central evaluation paradigm and, for the first time, systematically demonstrates that it satisfies core criteria from the philosophy of science—namely falsifiability, reproducibility, and predictive power. By constructing an evaluation framework that integrates causal analysis with mechanistic probing, the study delineates three functional pathways through which interpretability enables the identification of behavioral origins, detection of latent flaws, and prediction of potential failure modes. This approach advances model assessment beyond performance-oriented benchmarks toward a deeper understanding of underlying mechanisms.

This study addresses a critical limitation in existing natural language explanation methods for tabular data: their neglect of users’ inherent understandability of input features, which undermines explanation effectiveness. To bridge this gap, the work introduces, for the first time, a feature understandability scale into a cost-sensitive feature selection framework and proposes a joint optimization algorithm that simultaneously maximizes model classification accuracy and the understandability of selected features. Experimental results on two real-world datasets demonstrate that the proposed approach significantly enhances the surface understandability of generated explanations while maintaining high predictive performance. These findings validate the feasibility of co-optimizing accuracy and understandability, marking a pivotal shift in explainable AI—from design-driven paradigms toward alignment with user cognition.

This work addresses the lack of auditable evidence chains in existing computational pathology models, whose natural language explanations often consist of unverifiable correlational statements. The authors propose the first neuro-symbolic agent framework that uses SQL as its reasoning trace: an interpretable feature extraction module derives quantitative cellular attributes, which a reasoning agent then translates into executable SQL queries to aggregate visual evidence into structured results. A knowledge alignment agent subsequently validates these results against established pathological knowledge to support diagnostic conclusions. Evaluated on two histopathology visual question answering benchmarks, the method significantly enhances model interpretability and decision traceability, establishing—for the first time—a transparent, executable, and verifiable SQL-based evidence chain that directly links cellular measurements to diagnostic outcomes.

Current automated methods for evaluating interpretability struggle to keep pace with increasingly complex autonomous explanatory agents, particularly due to the subjectivity, opacity, and memory biases inherent in paradigms that aim to replicate human expert explanations. This work proposes a novel unsupervised intrinsic evaluation approach grounded in the functional interchangeability of model components and implements it within a large language model–driven autonomous research agent system that iteratively designs experiments and tests hypotheses in circuit analysis tasks. Experiments across six benchmark tasks show that while the system’s performance appears comparable to that of human experts, deeper analysis exposes fundamental flaws in replication-based evaluation. The proposed method effectively overcomes key limitations of traditional paradigms, offering a more reliable pathway for interpretability assessment.

Knowledge-intensive domains (e.g., surgery, astronomy, psychotherapy) demand that large language model explanations not only be logically coherent but also align with domain experts’ cognitive intuitions—yet existing evaluation methods emphasize superficial plausibility and lack quantitative measures of expert alignment. Method: We introduce T-FIX, the first benchmark to formalize “expert alignment” as a core interpretability metric, co-developed with domain experts across seven disciplines; it integrates textual explanations with feature-level interpretability analysis and defines quantifiable expert consistency metrics grounded in real-world clinical and research scenarios. Contribution/Results: T-FIX significantly enhances the credibility and practical utility of model explanations in professional contexts, establishing a novel paradigm for trustworthy AI by bridging the gap between algorithmic interpretability and domain-specific epistemic standards.