Adaptive Knowledge Transferring with Switching Dual-Student Framework for Semi-Supervised Medical Image Segmentation

2025-10-28

AI Summary

In semi-supervised medical image segmentation, teacher-student frameworks suffer from strong model coupling, unreliable pseudo-labels, and error accumulation. To address these issues, we propose the Switchable Dual-Student (SDS) framework, which abandons the fixed teacher role and instead employs two co-evolving student networks. SDS dynamically selects the superior model to generate high-quality pseudo-labels via a loss-aware exponential moving average (EMA) mechanism and enforces consistency regularization for robust knowledge feedback. The method is plug-and-play, introduces no additional parameters, and achieves significant improvements over state-of-the-art approaches across multiple 3D medical segmentation benchmarks—especially under extremely low labeling ratios (e.g., 1%–5%). Experimental results demonstrate substantial mitigation of model coupling and error propagation, validating SDS’s robustness and generalizability.

Technology Category

• Machine Learning: Semi-Supervised Learning
• Computer Vision: Segmentation
• Reasoning under Uncertainty: Graphical Models

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs

Abstract

Teacher-student frameworks have emerged as a leading approach in semi-supervised medical image segmentation, demonstrating strong performance across various tasks. However, the learning effects are still limited by the strong correlation and unreliable knowledge transfer process between teacher and student networks. To overcome this limitation, we introduce a novel switching Dual-Student architecture that strategically selects the most reliable student at each iteration to enhance dual-student collaboration and prevent error reinforcement. We also introduce a strategy of Loss-Aware Exponential Moving Average to dynamically ensure that the teacher absorbs meaningful information from students, improving the quality of pseudo-labels. Our plug-and-play framework is extensively evaluated on 3D medical image segmentation datasets, where it outperforms state-of-the-art semi-supervised methods, demonstrating its effectiveness in improving segmentation accuracy under limited supervision.

Problem

• Enhances unreliable knowledge transfer in teacher-student networks
• Improves pseudo-label quality via dynamic teacher-student collaboration
• Boosts semi-supervised medical image segmentation accuracy

Innovation

• Switching Dual-Student architecture enhances collaboration reliability
• Loss-Aware Exponential Moving Average improves pseudo-label quality
• Plug-and-play framework outperforms semi-supervised segmentation methods