SmoothOperator: Enhancing Representations for Fine-grained Open-set Recognition via Modulated Label Smoothing

2026-09-30

AI Summary

This study addresses the limitation of fixed label smoothing coefficients in open-set recognition, which fail to adapt to varying sample embedding quality and consequently lead to the misclassification of unknown classes. To overcome this, we propose SmoothOP, a plug-and-play module that reveals, for the first time, the structural properties of objective functions in spherical representation learning. This method reinterprets label smoothing as an alignment control knob and dynamically adjusts the smoothing coefficient based on sample saliency, thereby adaptively optimizing the trade-off between alignment and uniformity in the feature space. Experimental results demonstrate that SmoothOP achieves up to a 4.7% improvement in AUROC on semantic shift benchmarks, significantly outperforming multiple baseline methods.

Abstract

Open Set Recognition (OSR) aims to enable models to accurately classify known classes while rejecting samples from unseen classes. A key challenge in OSR lies in the inability to model the unbounded distribution of unknown classes during training, often leading to the misclassification of samples from these classes. Rather than modeling unknowns, recent work shapes the feature space so that known classes are compact and well separated, and spherical representation learning methods have achieved strong results this way. Label smoothing has been identified as one of the key drivers of this success, yet it applies the same coefficient to every training sample, regardless of how well each sample is already embedded. We show that the spherical representation learning objectives used in OSR share a single alignment--uniformity structure in which labels enter only through the alignment term. Label smoothing therefore acts as an alignment dial, and a fixed coefficient sets this dial to the same value for every sample. We propose a plug-in, SmoothOperator (SmoothOP), which sets the smoothing coefficient of each sample from its \textbf{prominence}, an embedding-space signal measuring how clearly the sample's own class stands out against its strongest competing class. Our method integrates into four existing spherical representation learning methods at minimal training overhead. SmoothOP assigns strong smoothing to samples with high prominence, which reduces their alignment and relaxes their pull. On the Semantic Shift Benchmark, SmoothOP-augmented variants generally outperform their base objectives across datasets, degrees of semantic shift, and OSR post-processors, with gains of up to 4.7\% in AUROC, OSCR, and closed-set accuracy.

Problem

• Open Set Recognition
• Label Smoothing
• Spherical Representation Learning
• Fine-grained Recognition

Innovation

• Open Set Recognition
• Label Smoothing
• Spherical Representation Learning
• Alignment-Uniformity
• Plug-in Module