SADe: Sparse-Atom Support Decontamination for Few-Shot Segmentation with Weak Support Annotations

2026-07-27

AI Summary

This work addresses the challenge of texture-similar interference and background noise introduced by weak supervision signals—such as bounding boxes or scribbles—in few-shot segmentation, which often corrupt support prototypes or visual prompts. To mitigate this, the authors propose SADe, a predictor-agnostic support decontamination layer that leverages atomic activation contrasts from sparse autoencoders as factor-level reliability cues to assess the trustworthiness of support image patches without requiring query information. A lightweight routing mechanism then generates clean support masks. SADe is designed as a plug-and-play module and consistently enhances downstream model performance: it achieves the highest mIoU in 6 out of 9 prompt configurations, outperforms raw support annotations in 70 out of 72 weakly supervised experiments, and significantly surpasses SAM3-derived masks under loose bounding box settings.

Abstract

Few-shot segmentation (FSS) commonly assumes clean pixel-level support masks, yet practical support supervision often uses boxes, scribbles, coarse masks, or pseudo-masks. These weak annotations may include texture-similar distractors and background context alongside the target, contaminating class prototypes or visual prompts before query prediction. We introduce SADe, a predictor-agnostic support decontamination layer that estimates the reliability of selected support patches without query information. Central to SADe is sparse autoencoder (SAE) atom evidence: dense similarity may respond to both target and texture-similar context, whereas contrasting atom activations inside and outside the weak-support region provides factor-level reliability cues. A lightweight router combines atom evidence with dense similarity and episode statistics to predict patch reliability and generate a cleaned support mask. Trained once on synthetic weak-support episodes from FSS-1000, the router is frozen for all target evaluations. The resulting mask supports standalone prediction or can be supplied to heterogeneous FSS models through native support interfaces without altering query-side inference. Under a matched weak-support protocol, SADe achieves the highest query mIoU in six of nine standalone prompt-shot combinations. With the same ProMi query head, it is within 0.03 mIoU of SAM3-derived masks under tight boxes and surpasses them by 11.17 and 19.49 points under box-r2 and box-r4, respectively. As a plug-in, SADe improves over raw support in 70 of 72 matched box-family comparisons across four frozen downstream models and two datasets. On point and scribble prompts, its average performance remains close to the corresponding raw-support baseline. Ablations and atom-removal controls show that atom evidence contributes reliability information beyond dense similarity.

Problem

• Few-Shot Segmentation
• Weak Support Annotations
• Support Decontamination
• Prototype Contamination
• Sparse Autoencoder

Innovation

• Sparse Autoencoder
• Support Decontamination
• Few-Shot Segmentation
• Weak Annotations
• Prompt Reliability