Contrastive Prompt Clustering for Weakly Supervised Semantic Segmentation

2025-08-23

AI Summary

Existing weakly supervised semantic segmentation (WSSS) methods under image-level labels overemphasize class separation while neglecting semantic correlations among related categories and lacking fine-grained discriminative capability. Method: We propose a contrastive prompt clustering framework: (1) leveraging large language models (LLMs) to mine inter-class semantic relationships and construct transferable category clustering priors; (2) designing a class-aware block-level contrastive loss and hierarchical contrastive learning mechanism to enhance intra-class consistency and inter-class separability under coarse-grained semantic guidance, while preserving fine-grained object boundaries; and (3) integrating pseudo-label optimization to refine segmentation accuracy. Contribution/Results: Our method achieves significant improvements over state-of-the-art approaches on PASCAL VOC 2012 and MS COCO 2014, effectively mitigating confusion among visually similar categories.

Technology Category

• Computer Vision: Segmentation
• Machine Learning: Semi-Supervised Learning
• Natural Language Processing: Sentence-level Semantics, Textual Inference, etc.

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Weakly Supervised Semantic Segmentation (WSSS) with image-level labels has gained attention for its cost-effectiveness. Most existing methods emphasize inter-class separation, often neglecting the shared semantics among related categories and lacking fine-grained discrimination. To address this, we propose Contrastive Prompt Clustering (CPC), a novel WSSS framework. CPC exploits Large Language Models (LLMs) to derive category clusters that encode intrinsic inter-class relationships, and further introduces a class-aware patch-level contrastive loss to enforce intra-class consistency and inter-class separation. This hierarchical design leverages clusters as coarse-grained semantic priors while preserving fine-grained boundaries, thereby reducing confusion among visually similar categories. Experiments on PASCAL VOC 2012 and MS COCO 2014 demonstrate that CPC surpasses existing state-of-the-art methods in WSSS.

Problem

• Addresses inter-class separation neglect in WSSS
• Reduces confusion among visually similar categories
• Enhances fine-grained discrimination with semantic priors

Innovation

• Uses LLMs for category clustering
• Introduces patch-level contrastive loss
• Leverages hierarchical semantic priors