Prototype-Based Image Prompting for Weakly Supervised Histopathological Image Segmentation

2025-03-15

AI Summary

Weakly supervised segmentation of histopathological images suffers from incomplete Class Activation Map (CAM) activations and failure of text-prompting methods due to inter-class homogeneity and intra-class heterogeneity. Method: This paper proposes a prototype-guided contrastive matching framework that constructs a class-specific prototype image bank—first integrating prototype learning with image-level prompting—and employs multi-prototype modeling to capture intra-class heterogeneity while introducing a contrastive matching loss to mitigate inter-class confusion. CAM generation and weakly supervised segmentation are jointly optimized. Contribution/Results: The method achieves significant improvements over state-of-the-art approaches on four major benchmarks—LUAD-HistoSeg, BCSS-WSSS, GCSS, and BCSS—establishing new state-of-the-art performance in weakly supervised histopathological image segmentation.

Technology Category

• Computer Vision: Segmentation
• Machine Learning: Semi-Supervised Learning
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Weakly supervised image segmentation with image-level labels has drawn attention due to the high cost of pixel-level annotations. Traditional methods using Class Activation Maps (CAMs) often highlight only the most discriminative regions, leading to incomplete masks. Recent approaches that introduce textual information struggle with histopathological images due to inter-class homogeneity and intra-class heterogeneity. In this paper, we propose a prototype-based image prompting framework for histopathological image segmentation. It constructs an image bank from the training set using clustering, extracting multiple prototype features per class to capture intra-class heterogeneity. By designing a matching loss between input features and class-specific prototypes using contrastive learning, our method addresses inter-class homogeneity and guides the model to generate more accurate CAMs. Experiments on four datasets (LUAD-HistoSeg, BCSS-WSSS, GCSS, and BCSS) show that our method outperforms existing weakly supervised segmentation approaches, setting new benchmarks in histopathological image segmentation.

Problem

• Addresses incomplete masks in weakly supervised histopathological image segmentation.
• Overcomes inter-class homogeneity and intra-class heterogeneity challenges.
• Improves accuracy of Class Activation Maps using prototype-based prompting.

Innovation

• Prototype-based image prompting for segmentation
• Clustering extracts multiple prototype features
• Contrastive learning improves CAM accuracy