Denoising Mutual Knowledge Distillation in Bi-Directional Multiple Instance Learning

2025-05-17

AI Summary

In digital pathology whole-slide image classification, multi-instance learning (MIL) suffers from inaccurate bag- and instance-level predictions and severe pseudo-label noise due to weak supervision. To address this, we propose a bidirectional denoising mutual knowledge distillation framework. Our method introduces, for the first time, a collaborative pseudo-label correction mechanism between bag-level and instance-level prediction pathways, integrating weak-to-strong generalization, multi-instance modeling, and knowledge distillation into a dual-path mutual feedback learning architecture. Evaluated on multiple public pathological datasets, our approach significantly improves bag-level classification accuracy and instance localization precision, while also enhancing instance mask quality and model robustness. It effectively narrows the performance gap between weakly supervised MIL and fully supervised learning.

Technology Category

• Machine Learning: Multi-instance/Multi-view Learning
• Computer Vision: Multi-modal Vision
• Multiagent Systems: Multiagent Learning

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Multiple Instance Learning is the predominant method for Whole Slide Image classification in digital pathology, enabling the use of slide-level labels to supervise model training. Although MIL eliminates the tedious fine-grained annotation process for supervised learning, whether it can learn accurate bag- and instance-level classifiers remains a question. To address the issue, instance-level classifiers and instance masks were incorporated to ground the prediction on supporting patches. These methods, while practically improving the performance of MIL methods, may potentially introduce noisy labels. We propose to bridge the gap between commonly used MIL and fully supervised learning by augmenting both the bag- and instance-level learning processes with pseudo-label correction capabilities elicited from weak to strong generalization techniques. The proposed algorithm improves the performance of dual-level MIL algorithms on both bag- and instance-level predictions. Experiments on public pathology datasets showcase the advantage of the proposed methods.

Problem

• Improving accuracy of bag- and instance-level classifiers in MIL
• Reducing noisy labels in Multiple Instance Learning methods
• Bridging gap between MIL and fully supervised learning

Innovation

• Bi-directional knowledge distillation for MIL
• Pseudo-label correction for noisy instances
• Weak to strong generalization enhancement