Problem
Research questions and friction points this paper is trying to address.
long-tailed recognition
class imbalance
tail classes
softmax classifier
preference issue
Class-Balanced Softmax: A Bayes Theory-Based Method for Long-Tailed Recognition
🤖 AI Summary
This work addresses the significant performance degradation of deep learning models on tail classes in long-tailed imbalanced datasets by introducing, for the first time, a Bayesian perspective to reveal and quantify the model’s “preference bias” toward head classes. Leveraging Bayesian theory and the assumption that class frequencies follow a power-law distribution, the authors propose a computationally efficient and easily integrable log-adjustment method to achieve class balance during training. A novel metric is introduced to evaluate preference bias, and extensive experiments on multiple large-scale long-tailed benchmarks demonstrate that the proposed approach substantially outperforms existing techniques—such as Balanced Softmax—while exhibiting strong scalability and practical utility.
📝 Abstract
Deep learning models using traditional softmax classifiers have achieved remarkable success in various classification tasks. However, their performance degrades significantly on imbalanced datasets. Although Balanced Softmax is widely adopted as a state-of-the-art rebalancing method, it possesses inherent limitations, such as yielding disproportionately lower testing accuracy for tail classes. To mitigate these shortcomings, we propose the Class-Balanced Softmax (CBS). Rooted in a theoretical Bayesian framework and a heuristic power-law assumption, the CBS is a simple logit adjustment that is computationally inexpensive and easily integrated into existing pipelines. Furthermore, we characterise a fundamental phenomenon in models trained on imbalanced data, termed the preference issue, wherein models exhibit higher training error and a larger generalisation gap for classes with limited data. To quantify this issue, we introduce a novel metric and demonstrate that CBS effectively mitigates the preference issue. Extensive experiments on large-scale benchmarks show that CBS is highly scalable and outperforms existing methods, including Balanced Softmax.
Innovation
Methods, ideas, or system contributions that make the work stand out.
Class-Balanced Softmax
Bayesian framework
long-tailed recognition
preference issue
logit adjustment
🔎 Similar Papers
No similar papers found.
💼 Related Jobs
