Weakly Supervised Concept Learning with Class-Level Priors for Interpretable Medical Diagnosis

2025-11-02

AI Summary

Medical interpretable AI faces two key challenges: prohibitive costs and impracticality of manual annotation of clinical concepts in real-world settings, and insufficient reliability of existing zero-shot vision-language models (VLMs) and concept generation methods. To address these, we propose PCP—a weakly supervised concept learning framework that leverages only class-level concept priors (e.g., “malignant” associated with clinical terms such as “irregular borders” and “pigmentation”) as supervision, eliminating the need for explicit concept annotations or large language model assistance. PCP employs a Prior-guided Concept Predictor, aligning predicted concept distributions with clinical reasoning logic via KL divergence minimization and enhancing concept discriminability through entropy regularization. On PH2 and WBCatt, PCP achieves over 33% improvement in concept F1-score versus zero-shot baselines. Across four medical datasets, its classification performance matches fully supervised concept bottleneck models (CBMs) and V-IP—marking the first demonstration of highly reliable, purely weakly supervised medical concept learning.

Technology Category

• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning
• Computer Vision: Interpretability, Explainability, and Transparency
• Machine Learning: Calibration & Uncertainty Quantification

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Web Mining and Content Analysis: Large pretrained models with web data
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Human-interpretable predictions are essential for deploying AI in medical imaging, yet most interpretable-by-design (IBD) frameworks require concept annotations for training data, which are costly and impractical to obtain in clinical contexts. Recent attempts to bypass annotation, such as zero-shot vision-language models or concept-generation frameworks, struggle to capture domain-specific medical features, leading to poor reliability. In this paper, we propose a novel Prior-guided Concept Predictor (PCP), a weakly supervised framework that enables concept answer prediction without explicit supervision or reliance on language models. PCP leverages class-level concept priors as weak supervision and incorporates a refinement mechanism with KL divergence and entropy regularization to align predictions with clinical reasoning. Experiments on PH2 (dermoscopy) and WBCatt (hematology) show that PCP improves concept-level F1-score by over 33% compared to zero-shot baselines, while delivering competitive classification performance on four medical datasets (PH2, WBCatt, HAM10000, and CXR4) relative to fully supervised concept bottleneck models (CBMs) and V-IP.

Problem

• Developing interpretable medical AI without costly concept annotations
• Improving reliability of domain-specific medical feature learning
• Enhancing concept prediction using class-level priors as weak supervision

Innovation

• Weakly supervised concept predictor without explicit supervision
• Leverages class-level concept priors as weak supervision
• Uses KL divergence and entropy regularization refinement