semantic segmentation

Visual activity recognition evaluation suffers from verb semantic ambiguity (e.g., synonymous verbs “brushing teeth” and “combing hair”) and image interpretation polysemy (e.g., “driving” vs. “operating”), causing conventional exact-match metrics—which rely on a single ground-truth label—to underestimate model robustness. To address this, we propose a novel evaluation framework based on vision-language clustering: leveraging the imSitu dataset, we construct verb sense clusters wherein semantically similar action descriptions are grouped, enabling multiple valid predictions to match a single image. Experiments show each image associates with an average of 2.8 sense clusters, confirming cluster validity. Our method significantly improves alignment with human judgments over exact matching and demonstrates superior discriminative power across diverse models. The core contribution is the first systematic integration of verb sense clustering into visual activity evaluation, yielding a more cognitively grounded and robust assessment paradigm.

This work addresses the challenge of recovering local visual semantics from CLIP’s global representations to support dense vision–language tasks, without introducing additional training or external models. The authors propose a training-free framework that uncovers the implicit local semantic accessibility within CLIP by analyzing the contribution of image patches to the attention output of the [CLS] token. They further construct a semantic–geodesic topological gate to calibrate affinities among image patches, enabling high-quality dense semantic reconstruction. Evaluated on eight zero-shot semantic segmentation benchmarks, the method achieves consistent improvements of 1.3–4.5 percentage points in mean Intersection-over-Union (mIoU) over the current best training-free approaches, demonstrating its effectiveness and strong generalization capability.

To address challenges in remote sensing (RS) image analysis—including annotation scarcity, multimodality, resolution variability, and scene heterogeneity—this paper proposes a unified representation learning framework for weakly supervised and few-shot learning. Methodologically, it introduces: (1) an anomaly-aware unsupervised object discovery mechanism; (2) a cross-dataset joint training paradigm to integrate heterogeneous unlabeled RS data; (3) an explicit-implicit coupled class-hierarchical modeling strategy for few-shot scene classification; and (4) multimodal contrastive learning to enhance cross-sensor semantic alignment. Evaluated on both natural-image benchmarks and diverse RS datasets (e.g., EuroSAT, UC Merced, DOTA), the framework achieves state-of-the-art performance in scene classification, object detection, and semantic segmentation under weakly supervised and few-shot settings. Results demonstrate consistent and significant improvements over existing baselines, validating its effectiveness for practical RS intelligent interpretation tasks.

Traditional object detection suffers from heavy reliance on labor-intensive manual annotations, poor generalization, and limited adaptability to novel categories and dynamic environments. To address these challenges, this work proposes an end-to-end fully automated detection pipeline. Methodologically, it introduces— for the first time—a unified framework integrating CLIP-driven open-vocabulary localization, diffusion model–enhanced feature representation, uncertainty-aware pseudo-label filtering, and an interactive human verification mechanism. This enables zero-shot category extension and closed-loop optimization with controllable annotation quality. Built upon a fine-tuned YOLOv8 backbone, the method achieves 92% of the full-supervision state-of-the-art mAP on COCO and LVIS using only 15% of the manual annotations required by conventional approaches. The proposed pipeline significantly reduces annotation cost while substantially improving cross-domain generalization capability.

Open-world image region semantic recognition faces challenges including high computational overhead, poor noise robustness, and insufficient contextual modeling. This paper proposes a lightweight, fine-tuning-free paradigm: freezing the Segment Anything Model (SAM) for localization and CLIP for vision-language representation, while training only a lightweight attention-based fusion module. We further introduce a zero-shot region alignment mechanism—marking the first approach to achieve efficient synergy between precise localization and semantic understanding. The method balances efficiency, robustness, and contextual awareness. On the LVIS val set, it achieves a 2.9 mAP gain over GLIP-L, with a 13.1 AP improvement on rare categories. It also yields +2.5 AP on ODinW and +11.0 AP on LVIS minival. Training completes in one day on 8×V100 GPUs using 3 million samples.

Existing object detectors often learn task-driven features that rely on shortcut correlations, failing to adequately capture the underlying annotation structure, which limits their generalization, interpretability, and robustness under task shifts or sparse supervision. To address this, this work proposes an annotation-guided feature enhancement framework that explicitly integrates geometric annotation priors into feature learning for the first time. By constructing a dense spatial feature grid and injecting it into the backbone network—where it fuses with the feature pyramid—the method steers region proposal and detection heads toward representations better aligned with annotation structure. Evaluated on wildlife and remote sensing datasets, the approach significantly improves object focus, reduces background sensitivity, and demonstrates superior generalization and data efficiency in weakly supervised and unseen-task settings.

This work addresses the overestimation of semantic shifts between large-scale natural image datasets in existing supervised classification methods, which often rely on non-semantic artifacts such as resolution rather than genuine semantic content. To rectify this, the authors propose the first unsupervised semantic clustering framework that leverages semantic features extracted from foundation vision models to directly assess inter-dataset semantic separability without requiring labels. Through carefully designed controlled experiments, they demonstrate that the high classification accuracy reported by conventional approaches primarily stems from non-semantic confounders. When applied to mainstream web-scale datasets, their method yields clustering performance near random chance, providing strong evidence that previously reported semantic discrepancies have been substantially exaggerated.

This work investigates how semantic priors influence semantic localization performance and robustness within self-supervised contrastive learning frameworks, particularly focusing on noise suppression and selective attention to discriminative landmarks rather than generic clutter. We propose an implicit weighting mechanism based on semantic-class ablation, revealing the model’s intrinsic tendency to downweight high-frequency, low-discriminative objects. To validate interpretability under visual and structural variations, we integrate gradient-based attribution (e.g., integrated gradients), attention visualization, semantic scene graph modeling, and posterior introspection analysis. Experimental results demonstrate that the learned location representations exhibit both noise resilience and semantic saliency, enabling stable and interpretable cross-view matching across diverse challenging scenarios—including occlusion, viewpoint shifts, and environmental degradation.

Existing vision foundation models lack a unified, fine-grained evaluation protocol for structured object understanding, particularly suffering from inconsistent evaluation setups and insufficient supervision in part-level semantic correspondence tasks across instances and categories. This work proposes the SOCO benchmark, which establishes the first unified framework for semantic object correspondence, featuring million-scale functional keypoint annotations and accompanying textual descriptions across 100 object categories. Systematic evaluation of vision and vision-language foundation models reveals that visual backbones exhibit strong semantic structure awareness but limited cross-category generalization; large vision-language models outperform purely visual approaches in text-guided localization; and crucially, semantic correspondence performance serves as a more effective predictor than ImageNet accuracy for downstream tasks such as segmentation, tracking, and 3D pose estimation.

This work addresses the challenge of insufficient accuracy in segmenting fine structures and object boundaries—particularly in resource-constrained settings—by proposing FoR-Net, a lightweight architecture for semantic segmentation. FoR-Net incorporates a region-focused inductive bias that selectively enhances information-rich and challenging regions through importance map prediction and a Top-K activation mechanism. It further aggregates multi-scale spatial context via parallel convolutional branches. Evaluated on the Cityscapes benchmark under standard training protocols and limited computational resources, FoR-Net achieves competitive overall performance while significantly improving segmentation consistency in difficult regions.