Induce to Empower: Improving Lightweight Baselines via Foundation Model Induction for Generalized Polyp Segmentation

2026-07-19

AI Summary

This work addresses the limited generalization of existing lightweight polyp segmentation models and the prohibitive computational cost of foundation models that hinder real-time clinical deployment. To bridge this gap, the authors propose Lite-Polyp Inductor, a novel framework that, for the first time, injects complementary prior knowledge from multiple foundation models—DINOv2, SAM, and OneFormer—into a lightweight U-Net architecture through prototype representations. The approach further enhances semantic and boundary representation by integrating reconstruction-based supervision with a Transformer-based feature fusion mechanism. Evaluated on five benchmark datasets, the method achieves substantial improvements in both segmentation accuracy and generalization while maintaining low computational overhead, thereby enabling efficient and robust polyp segmentation suitable for clinical settings.

Abstract

Automated polyp segmentation in colonoscopy continues to pose challenges due to substantial appearance variations and indistinct polyp boundaries. Although emerging foundation models (FMs) such as DINOv2, SAM, and OneFormer, demonstrate remarkable generalization capabilities, their direct transfer to the polyp segmentation task and deployment in real-time clinical settings are difficult due to lack of large-scale labeled data and high computational demands. In addition, adopting multiple FMs together raises concerns, even though they encode complementary semantic and structural information. While lightweight models, including U-Net, PraNet and U-Net++, are computationally efficient, they often struggle to generalize across datasets due to limited representational capacity. To address this gap, we propose Lite-Polyp Inductor (Lite-Pi), a novel foundation model induction framework that significantly enhances lightweight polyp segmentation baselines. Our proposed framework generates FM-specific prototype representations and aligns them semantically with the corresponding foundation model priors through reconstruction-based supervision. Subsequently, transformer-based fusion is introduced to highlight the polyp relevant representations, including salient boundary information, while preserving complementary semantic cues. Extensive experiments across five polyp segmentation benchmark datasets demonstrate that Lite-π significantly improves lightweight baselines, achieving superior generalization performance with minimal computational overhead and thereby, offering a practical solution for generalized polyp segmentation. Our code is available at GitHub. https://github.com/lostinrepo/Lite-Pi

Problem

• polyp segmentation
• foundation models
• lightweight models
• generalization
• colonoscopy

Innovation

• foundation model induction
• lightweight segmentation
• polyp segmentation
• prototype representation
• transformer-based fusion