Problem
Research questions and friction points this paper is trying to address.
Weakly Supervised Semantic Segmentation
Boundary Refinement
Pseudo-labels
Class Activation Maps
DenseCRF
CRF Loss is How Networks Should Learn Boundaries in Weakly Supervised Segmentation
🤖 AI Summary
This study addresses the issues of error fixation caused by hard labels and insufficient boundary precision in weakly supervised semantic segmentation by proposing the DS-CRF framework. Inspired by Conditional Random Fields, this method decouples unary and pairwise potential supervision signals. Specifically, it leverages DINO text CAMs to provide soft classification supervision and incorporates SAM to extract boundary information, optimizing training via a custom CRF loss function. This design effectively prevents the error amplification typically induced by conventional pseudo-label fusion while preserving prediction uncertainty to enhance boundary learning. Experimental results demonstrate that the proposed framework achieves 56.5% mIoU on the MS COCO dataset, establishing a new state-of-the-art performance.
📝 Abstract
Weakly Supervised Semantic Segmentation (WSSS) learns pixel-level predictions from image-level tags. Recent work focuses on improving coarse CAMs extracted from large vision-language models (commonly CLIP), but does little to improve their accuracy along segment boundaries. That job is instead delegated to a post-processing method like DenseCRF. However, because DenseCRF relies on low-level colour cues, it can flip correct labels to incorrect ones when neighbouring pixels share similar colours. SAM has recently been adopted as a natural alternative, yet it simply takes on DenseCRF's role as an intermediate"refinement"step that outputs one-hot pseudo-labels in prior work. By discarding the valuable uncertainty in CAMs, these one-hot pseudo-labels turn borderline errors into confidently wrong targets. Our key insight is that CAMs should supervise training alongside SAM boundaries, each through its own loss, rather than being fused together into a single hard target. Inspired by CRF potentials, we propose a framework that disentangles soft pseudo-labels as unary supervision and binary edge maps as pairwise supervision. We realize our framework in a single-stage model, DS-CRF, using CAMs from dino.txt and boundaries from SAM. DS-CRF sets a new state-of-the-art of 56.5% mIoU on MS COCO.
Innovation
Methods, ideas, or system contributions that make the work stand out.
Weakly Supervised Semantic Segmentation
CRF Loss
Soft Pseudo-labels Disentanglement
SAM Boundaries
Single-stage Model
🔎 Similar Papers
No similar papers found.
