Problem
Research questions and friction points this paper is trying to address.
Open-Vocabulary Semantic Segmentation
Domain Adaptation
Preference Learning
Prompt Disagreement
Preference-Guided Adaptation for Open-Vocabulary Semantic Segmentation via Prompt Disagreement
🤖 AI Summary
This study addresses the performance degradation of open-vocabulary semantic segmentation in specialized domains caused by the scarcity of dense annotations. To this end, it proposes a Region-wise Local Preference Optimization (RLPO) framework. This method leverages prompt divergence to mine local preferences, replacing pixel-level mask supervision with binary preferences and incorporating consistency regularization for efficient model adaptation. The core innovation lies in enabling cross-domain transfer without requiring dense annotations while maintaining robustness against noisy preferences. Experimental results demonstrate that the proposed approach significantly improves the segmentation performance of various backbone networks on the MESS benchmark.
📝 Abstract
Open-vocabulary semantic segmentation (OVSS) enables pixel-level prediction over arbitrary text-specified vocabularies and has shown strong generalization on common benchmarks. However, OVSS performance often degrades in specialized domains such as medical imaging, remote sensing, and industrial inspection, where dense pixel-level masks for adaptation are costly to obtain and require domain-specific expertise. We propose a preference-guided adaptation framework that replaces dense mask supervision with binary preferences. We observe that different prompt templates produce systematically different segmentations for the same image, a phenomenon we call prompt disagreement, and we repurpose it as a built-in source of preference supervision. Building on this, we mine localized preference queries from regions of high cross-template uncertainty, and adapt the OVSS model with Region-Localized Preference Optimization (RLPO) together with consistency regularization that stabilizes updates outside the queried region. Across extensive experiments on the MESS benchmark, the proposed method achieves consistent gains across diverse OVSS backbones without any pixel-level annotation, and remains effective under noisy preferences. Our code is available at https://github.com/blue-531/pref-ovss.
Innovation
Methods, ideas, or system contributions that make the work stand out.
Open-Vocabulary Semantic Segmentation
Preference-Guided Adaptation
Prompt Disagreement
Region-Localized Preference Optimization
Consistency Regularization
🔎 Similar Papers
No similar papers found.
