Prototype-Based Multiple Instance Learning for Gigapixel Whole Slide Image Classification

2025-03-11

AI Summary

Current multiple instance learning (MIL) models for whole-slide image (WSI) analysis rely heavily on attention mechanisms, yielding weak interpretability and non-editable predictions—failing to meet clinical requirements for trustworthiness. To address this, we propose ProtoMIL, the first framework that deeply integrates prototype learning with MIL for interpretable WSI analysis. ProtoMIL employs a sparse autoencoder to automatically discover semantically coherent visual prototypes; predictions are transparently formulated as linear combinations of these prototypes, enabling real-time clinician intervention and correction. This ensures the model is “right for the right reasons,” overcoming the black-box nature and immutability of attention-based explanations. ProtoMIL achieves state-of-the-art classification accuracy on two major pathology benchmarks while delivering concept-level, human-understandable interpretations. Empirical results demonstrate that manual removal of irrelevant prototypes significantly improves model robustness and diagnostic credibility.

Technology Category

• Computer Vision: Interpretability, Explainability, and Transparency
• Machine Learning: Multimodal Learning
• Natural Language Processing: Interpretability, Analysis, and Evaluation of NLP Models

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Multiple Instance Learning (MIL) methods have succeeded remarkably in histopathology whole slide image (WSI) analysis. However, most MIL models only offer attention-based explanations that do not faithfully capture the model's decision mechanism and do not allow human-model interaction. To address these limitations, we introduce ProtoMIL, an inherently interpretable MIL model for WSI analysis that offers user-friendly explanations and supports human intervention. Our approach employs a sparse autoencoder to discover human-interpretable concepts from the image feature space, which are then used to train ProtoMIL. The model represents predictions as linear combinations of concepts, making the decision process transparent. Furthermore, ProtoMIL allows users to perform model interventions by altering the input concepts. Experiments on two widely used pathology datasets demonstrate that ProtoMIL achieves a classification performance comparable to state-of-the-art MIL models while offering intuitively understandable explanations. Moreover, we demonstrate that our method can eliminate reliance on diagnostically irrelevant information via human intervention, guiding the model toward being right for the right reason. Code will be publicly available at https://github.com/ss-sun/ProtoMIL.

Problem

• Develops interpretable MIL model for WSI analysis
• Enables human intervention in model decision process
• Improves classification transparency and diagnostic relevance

Innovation

• Uses sparse autoencoder for interpretable concepts
• Represents predictions via linear concept combinations
• Allows human intervention to alter input concepts