Training-Free Fine-Grained Semantic Segmentations in Low Data Regimes: A FungiTastic Baseline

2026-05-21

AI Summary

This work addresses the challenges of precise localization and discrimination among visually similar classes in fine-grained semantic segmentation under low-data regimes, particularly for fungal images exhibiting long-tailed distributions and varying acquisition conditions. To tackle these issues, the authors propose a training-free, two-stage decoupled framework: first, category-agnostic masks are generated using SAM3 guided by coarse-class prompts; then, fine-grained labels are assigned via prototype matching in the DINOv2 embedding space, augmented with simple feature-space transformations to enhance classification performance. This approach establishes the first effective baseline for low-data fine-grained segmentation, demonstrating superior performance across settings ranging from one-shot to hundreds of samples, while offering strong scalability and low computational cost.

Abstract

Fine-grained semantic segmentation requires both precise localization and discrimination between visually similar classes. In FungiTastic, this problem is further complicated by a long-tailed distribution and strong variation in image acquisition conditions. We propose a training-free two-stage framework that decouples segmentation from classification. SAM3 first produces class-agnostic mushroom masks using macro-taxonomic prompts, and DINOv3 then assigns fine-grained labels through prototype matching in the embedding space. To improve this stage, we apply a simple transformation of the DINOv3 feature space that improves prototype-based classification. Compared with class-specific prompting, our approach is more scalable and keeps the segmentation cost low. We report results from one-shot to few-hundred-shot regimes, providing, to the best of our knowledge, the first baseline for fine-grained semantic segmentation in low-data settings.

Problem

• fine-grained semantic segmentation
• low-data regime
• long-tailed distribution
• visual similarity
• fungi

Innovation

• training-free
• fine-grained semantic segmentation
• prototype matching
• feature space transformation
• low-data regime