Frustratingly Easy Feature Reconstruction for Out-of-Distribution Detection

2025-09-02

AI Summary

This paper addresses out-of-distribution (OOD) detection in privacy-sensitive settings where no training data is accessible. We propose a post-hoc method based on orthogonal decomposition of classifier weights. Without requiring any original training samples, our approach constructs a low-dimensional subspace spanned by the weight vectors of known classes and performs OOD detection via the reconstruction error of input features projected onto this subspace. Our key contribution is the first design of a data-free feature reconstruction mechanism grounded in subspace projection and reconstruction—enabling effective OOD discrimination while strictly preserving data privacy. Extensive experiments on standard OOD benchmarks—including CIFAR-10/100 and ImageNet-O—demonstrate state-of-the-art performance. The method is computationally efficient, conceptually simple, and fully reproducible; the implementation is publicly available.

Technology Category

• Machine Learning: Privacy
• Computer Vision: Bias, Fairness & Privacy
• Data Mining & Knowledge Management: Anomaly/Outlier Detection

Application Category

• Security and Privacy: Data transparency and provenance
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• User Modeling, Personalization and Recommendation: User privacy protection in personalized systems

Abstract

Out-of-distribution (OOD) detection helps models identify data outside the training categories, crucial for security applications. While feature-based post-hoc methods address this by evaluating data differences in the feature space without changing network parameters, they often require access to training data, which may not be suitable for some data privacy scenarios. This may not be suitable in scenarios where data privacy protection is a concern. In this paper, we propose a simple yet effective post-hoc method, termed Classifier-based Feature Reconstruction (ClaFR), from the perspective of subspace projection. It first performs an orthogonal decomposition of the classifier's weights to extract the class-known subspace, then maps the original data features into this subspace to obtain new data representations. Subsequently, the OOD score is determined by calculating the feature reconstruction error of the data within the subspace. Compared to existing OOD detection algorithms, our method does not require access to training data while achieving leading performance on multiple OOD benchmarks. Our code is released at https://github.com/Aie0923/ClaFR.

Problem

• Detects out-of-distribution data without training data access
• Addresses data privacy concerns in OOD detection methods
• Uses feature reconstruction error for OOD scoring

Innovation

• Classifier-based Feature Reconstruction method
• Orthogonal decomposition of classifier weights
• Feature reconstruction error for OOD scoring