SCS-SupCon: Sigmoid-based Common and Style Supervised Contrastive Learning with Adaptive Decision Boundaries

2025-12-17

AI Summary

In fine-grained image classification, the minimal inter-class differences and substantial intra-class variations hinder the discriminative power of supervised contrastive learning. To address this, we propose a Sigmoid-based supervised contrastive learning framework. Our key contributions are: (1) a learnable temperature- and bias-parameterized Sigmoid pairwise contrastive loss that enables adaptive decision boundary adjustment; and (2) the first integration of explicit style-distance constraints into supervised contrastive learning, jointly enhancing discrimination and content-style disentanglement. The method is compatible with both CNN and Transformer backbones. On CIFAR-100 with ResNet-50, it achieves a 3.9% absolute improvement in top-1 accuracy over SupCon; on fine-grained benchmarks—CUB200 and Stanford Dogs—it outperforms CS-SupCon by 0.4–3.0 percentage points. Moreover, it significantly improves model robustness and generalization.

Technology Category

• Computer Vision: Learning & Optimization for CV
• Machine Learning: Semi-Supervised Learning
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Web Mining and Content Analysis: Large pretrained models with web data
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs

Abstract

Image classification is hindered by subtle inter-class differences and substantial intra-class variations, which limit the effectiveness of existing contrastive learning methods. Supervised contrastive approaches based on the InfoNCE loss suffer from negative-sample dilution and lack adaptive decision boundaries, thereby reducing discriminative power in fine-grained recognition tasks. To address these limitations, we propose Sigmoid-based Common and Style Supervised Contrastive Learning (SCS-SupCon). Our framework introduces a sigmoid-based pairwise contrastive loss with learnable temperature and bias parameters to enable adaptive decision boundaries. This formulation emphasizes hard negatives, mitigates negative-sample dilution, and more effectively exploits supervision. In addition, an explicit style-distance constraint further disentangles style and content representations, leading to more robust feature learning. Comprehensive experiments on six benchmark datasets, including CUB200-2011 and Stanford Dogs, demonstrate that SCS-SupCon achieves state-of-the-art performance across both CNN and Transformer backbones. On CIFAR-100 with ResNet-50, SCS-SupCon improves top-1 accuracy over SupCon by approximately 3.9 percentage points and over CS-SupCon by approximately 1.7 points under five-fold cross-validation. On fine-grained datasets, it outperforms CS-SupCon by 0.4--3.0 points. Extensive ablation studies and statistical analyses further confirm the robustness and generalization of the proposed framework, with Friedman tests and Nemenyi post-hoc evaluations validating the stability of the observed improvements.

Problem

• Addresses subtle inter-class differences and large intra-class variations in image classification
• Mitigates negative-sample dilution and lack of adaptive decision boundaries in contrastive learning
• Improves discriminative power for fine-grained recognition tasks across multiple datasets

Innovation

• Sigmoid-based contrastive loss with learnable parameters
• Adaptive decision boundaries to emphasize hard negatives
• Style-distance constraint disentangles style and content representations