Explainable AI and the Scientific Method: Interpretability-Guided Knowledge Discovery

2024-06-15

AI Summary

This paper addresses the challenge of AI’s “black-box” nature impeding expert collaboration in scientific discovery during the AI era. We propose “Interpretability-Guided Exploration” (IGE), a novel paradigm that redefines interpretability as a cognitive interface between scientists and AI—specifically, as *scientifically oriented explainable AI*—to jointly support divergent hypothesis generation and convergent knowledge validation. Methodologically, IGE integrates intrinsically interpretable models, attribution techniques (e.g., SHAP, LIME), generative AI–augmented reasoning, and cross-disciplinary workflow modeling. We establish a formal theoretical framework and collaborative pathway for IGE, providing an actionable methodology for AI-driven hypothesis formulation, experimental design, and iterative knowledge refinement. The approach significantly enhances AI’s substantive contribution to scientific exploration in the pre-AGI stage, bridging the gap between technical interpretability and domain-specific scientific reasoning.

Technology Category

• Humans and AI: Explainable AI (XAI) for Human Understanding
• Philosophy and Ethics of AI: Accountability, Interpretability & Explainability
• Machine Learning: Transparent, Interpretable, Explainable ML

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Search and Retrieval-Augmented AI: Agentic search

Abstract

The scientific method is the cornerstone of human progress across all branches of the natural and applied sciences, from understanding the human body to explaining how the universe works. The scientific method is based on identifying systematic rules or principles that describe the phenomenon of interest in a reproducible way that can be validated through experimental evidence. In the era of generative artificial intelligence, there are discussions on how AI systems may discover new knowledge. We argue that human complex reasoning for scientific discovery remains of vital importance, at least before the advent of artificial general intelligence. Yet, AI can be leveraged for scientific discovery via explainable AI. More specifically, knowing the `principles' the AI systems used to make decisions can be a point of contact with domain experts and scientists, that can lead to divergent or convergent views on a given scientific problem. Divergent views may spark further scientific investigations leading to interpretability-guided explanations (IGEs), and possibly to new scientific knowledge. We define this field as Explainable AI for Science, where domain experts -- potentially assisted by generative AI -- formulate scientific hypotheses and explanations based on the interpretability of a predictive AI system.

Problem

• Interpretability-guided knowledge discovery
• Explainable AI for scientific hypotheses
• AI principles for scientific collaboration

Innovation

• Explainable AI for Science
• Interpretability-Guided Knowledge Discovery
• Generative AI-assisted hypotheses