Level, Sharpness, and Corpus: Why Zero-Shot OOD Detector Rankings Do Not Transfer

2026-07-29

AI Summary

Existing zero-shot out-of-distribution (OOD) detectors exhibit non-transferable performance rankings across deployment scenarios, undermining the reliability of benchmark evaluations. Through a large-scale portability audit encompassing 17 in-distribution datasets, three vision-language models, and seven OOD detectors, this work reveals pervasive rank reversal phenomena. To address this, the authors propose CEG, a detector-agnostic framework that fuses complementary evidence—absolute match confidence, relative/spatial sharpness, and semantic coverage—via a non-compensatory strategy, without requiring OOD samples, external corpora, or learned fusion mechanisms. Experiments demonstrate that CEG substantially enhances robustness, reducing the FPR95 of GL-MCM and MCM from 38.1 and 42.6 to 28.8 and 30.5, respectively, thereby effectively mitigating detector sensitivity to deployment environments.

Abstract

Selecting a zero-shot out-of-distribution (OOD) detector for a new deployment is typically based on benchmark rankings, implicitly assuming that the highest-ranked detector will transfer across domains. We show that this assumption does not hold. Through a controlled portability audit across seventeen in-distribution datasets, three vision-language models, and seven representative zero-shot OOD detectors, we find that detector rankings reverse across deployments, every detector exceeds $80\%$ FPR95 on at least one domain, and the preferred detector depends on both the in-distribution data and the underlying VLM. We trace these reversals to complementary evidence channels in vision-language logits. Corpus-free detectors rely on different combinations of absolute match level and relative or spatial sharpness, while WordNet-based methods additionally depend on external semantic coverage. A simple proposition shows that level and sharpness cannot generally be recovered from one another, explaining why no single detector transfers reliably across deployments. Motivated by this diagnosis, we introduce the Complementary Evidence Guard (CEG), a detector-agnostic wrapper that preserves complementary evidence through a non-compensatory fusion of the base detector, level, and sharpness using only empirical in-distribution percentiles. Controls replacing these channels with entropy, logit variance, or random noise do not reproduce the gains. Without OOD samples, auxiliary corpora, or learned fusion, CEG reduces detector sensitivity and improves GL-MCM from $38.1$ to $28.8$ and MCM from $42.6$ to $30.5$ family-balanced FPR95.

Problem

• zero-shot OOD detection
• detector transferability
• ranking inconsistency
• vision-language models
• distribution shift

Innovation

• zero-shot OOD detection
• evidence complementarity
• level and sharpness
• detector portability
• Complementary Evidence Guard