The Power of Certainty: How Confident Models Lead to Better Segmentation

2025-07-14

AI Summary

Colonoscopy polyp detection and segmentation models often suffer from overfitting due to excessive parameter counts and biased training data, resulting in poor cross-center generalization. To address this, we propose a lightweight confidence-driven self-distillation framework that requires no external teacher model or additional storage overhead. During training, sample-wise confidence scores are dynamically computed and used to adaptively weight supervision signals; concurrently, intra-batch iterative contrastive learning and dynamic loss optimization are integrated. Our method significantly enhances model robustness and generalization: it achieves state-of-the-art Dice scores and superior cross-domain performance on multi-center benchmarks, with zero inference-time computational overhead. The core innovation lies in the first integration of dynamic confidence modeling with intra-model self-distillation—enabling efficient, resource-light, and highly generalizable polyp segmentation.

Technology Category

• Computer Vision: Segmentation
• Search and Optimization: Learning to Search
• Machine Learning: Calibration & Uncertainty Quantification

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Deep learning models have been proposed for automatic polyp detection and precise segmentation of polyps during colonoscopy procedures. Although these state-of-the-art models achieve high performance, they often require a large number of parameters. Their complexity can make them prone to overfitting, particularly when trained on biased datasets, and can result in poor generalization across diverse datasets. Knowledge distillation and self-distillation are proposed as promising strategies to mitigate the limitations of large, over-parameterized models. These approaches, however, are resource-intensive, often requiring multiple models and significant memory during training. We propose a confidence-based self-distillation approach that outperforms state-of-the-art models by utilizing only previous iteration data storage during training, without requiring extra computation or memory usage during testing. Our approach calculates the loss between the previous and current iterations within a batch using a dynamic confidence coefficient. To evaluate the effectiveness of our approach, we conduct comprehensive experiments on the task of polyp segmentation. Our approach outperforms state-of-the-art models and generalizes well across datasets collected from multiple clinical centers. The code will be released to the public once the paper is accepted.

Problem

• Reducing overfitting in polyp segmentation models
• Minimizing resource use in self-distillation methods
• Improving generalization across diverse clinical datasets

Innovation

• Confidence-based self-distillation for efficient training
• Dynamic confidence coefficient for loss calculation
• Utilizes previous iteration data without extra resources