Reducing Variability of Multiple Instance Learning Methods for Digital Pathology

2025-06-30

AI Summary

In digital pathology, multi-instance learning (MIL) models exhibit substantial performance instability—AUC varies by 10–15 points across independent runs—primarily due to stochasticity in weight initialization, mini-batch ordering, and learning rate scheduling, severely undermining method comparability and reproducibility. To address this, we propose a multi-fidelity model ensembling strategy: during early training, we dynamically select high-potential, stable submodels based on validation-set performance and fuse their predictions via weighted averaging. Our approach incurs no additional computational overhead and is agnostic to MIL architecture or initialization scheme. Extensive evaluation across two public datasets—comprising over 2,000 experimental runs—demonstrates that our method reduces AUC variance by over 60% (measured by standard deviation), accelerates hyperparameter optimization, and exhibits strong generalization across diverse MIL models and datasets.

Technology Category

• Machine Learning: Multi-instance/Multi-view Learning
• Computer Vision: Multi-modal Vision
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Digital pathology has revolutionized the field by enabling the digitization of tissue samples into whole slide images (WSIs). However, the high resolution and large size of WSIs present significant challenges when it comes to applying Deep Learning models. As a solution, WSIs are often divided into smaller patches with a global label ( extit{i.e., diagnostic}) per slide, instead of a (too) costly pixel-wise annotation. By treating each slide as a bag of patches, Multiple Instance Learning (MIL) methods have emerged as a suitable solution for WSI classification. A major drawback of MIL methods is their high variability in performance across different runs, which can reach up to 10-15 AUC points on the test set, making it difficult to compare different MIL methods reliably. This variability mainly comes from three factors: i) weight initialization, ii) batch (shuffling) ordering, iii) and learning rate. To address that, we introduce a Multi-Fidelity, Model Fusion strategy for MIL methods. We first train multiple models for a few epochs and average the most stable and promising ones based on validation scores. This approach can be applied to any existing MIL model to reduce performance variability. It also simplifies hyperparameter tuning and improves reproducibility while maintaining computational efficiency. We extensively validate our approach on WSI classification tasks using 2 different datasets, 3 initialization strategies and 5 MIL methods, for a total of more than 2000 experiments.

Problem

• Reducing performance variability in Multiple Instance Learning for pathology
• Addressing high AUC fluctuations in MIL methods due to initialization
• Improving reproducibility and stability of WSI classification models

Innovation

• Multi-Fidelity Model Fusion strategy
• Averages stable models from validation
• Reduces MIL performance variability effectively