Revisiting Logit Distributions for Reliable Out-of-Distribution Detection

2025-10-22

AI Summary

To address the insufficient reliability of out-of-distribution (OOD) detection in open-world settings, this paper proposes LogitGap—a lightweight, training-free method that leverages discriminative structural information inherent in the logit space of classification models. The core idea is twofold: (1) defining LogitGap as the difference between the top-two logits, which significantly enhances separability between in-distribution (ID) and OOD samples in logit space; and (2) modeling logit distributions via extreme value theory while adaptively selecting the most discriminative logit subset based on information density—requiring no auxiliary training or architectural modification. LogitGap is broadly applicable to both vision-language and pure-vision models. It achieves state-of-the-art performance across multiple standard OOD benchmarks—including CIFAR-10/100 and ImageNet-O—demonstrating strong robustness, generalization, and computational efficiency.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Fact-Checking / Misinformation Detection (NLP Focus)

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Out-of-distribution (OOD) detection is critical for ensuring the reliability of deep learning models in open-world applications. While post-hoc methods are favored for their efficiency and ease of deployment, existing approaches often underexploit the rich information embedded in the model's logits space. In this paper, we propose LogitGap, a novel post-hoc OOD detection method that explicitly exploits the relationship between the maximum logit and the remaining logits to enhance the separability between in-distribution (ID) and OOD samples. To further improve its effectiveness, we refine LogitGap by focusing on a more compact and informative subset of the logit space. Specifically, we introduce a training-free strategy that automatically identifies the most informative logits for scoring. We provide both theoretical analysis and empirical evidence to validate the effectiveness of our approach. Extensive experiments on both vision-language and vision-only models demonstrate that LogitGap consistently achieves state-of-the-art performance across diverse OOD detection scenarios and benchmarks. Code is available at https://github.com/GIT-LJc/LogitGap.

Problem

• Enhancing separability between in-distribution and out-of-distribution samples
• Exploiting logit space information for reliable OOD detection
• Developing efficient post-hoc methods without requiring retraining

Innovation

• Uses maximum and remaining logits relationship
• Identifies informative logits automatically
• Training-free strategy for OOD detection