prototypical contrastive learning

Supervised contrastive learning (SupCL) suffers from intra-class collapse—where embeddings of same-class samples become excessively clustered and lack discriminability—due to improper loss weighting. Method: We propose the Simplex-to-Simplex Embedding Model (SSEM), a theoretical framework grounded in simplex geometry that rigorously characterizes the optimal intra-class embedding distribution. It yields interpretable, computationally tractable guidelines for hyperparameter selection. Contribution/Results: Through theoretical modeling, geometric analysis, and loss optimization—validated on synthetic data and real-world benchmarks (CIFAR-10/100, ImageNet-LT)—SSEM significantly alleviates intra-class collapse, enhances embedding compactness, and improves inter-class separability. To our knowledge, this is the first geometry-driven, interpretable mechanism for *preventing* collapse in SupCL, bridging theoretical insight with practical efficacy.

To address insufficient generalization in few-shot image classification caused by scarce support samples for novel classes, this paper proposes a Transformer-based prototypical relational modeling method. The approach integrates prototypical learning with meta-training without auxiliary modules. Its core innovations are: (i) the first explicit modeling of structured relationships among class prototypes using a Transformer architecture; and (ii) a lightweight, parameter-free contrastive learning mechanism that jointly optimizes prototype discriminability in an end-to-end manner. Evaluated on miniImageNet, the method achieves state-of-the-art accuracy of 97.07% (5-way 5-shot) and 90.88% (5-way 1-shot), surpassing prior art by 0.57% and 6.84%, respectively. These results significantly advance the performance frontier of few-shot classification.

To address catastrophic forgetting in class-incremental learning, this paper proposes a Bayesian-driven prototype contrastive learning framework. The method introduces Bayesian uncertainty modeling into the prototype contrastive loss for the first time, enabling dynamic, adaptive weighting between cross-entropy and contrastive losses. It establishes a prototype-level contrastive learning paradigm tailored to incremental settings, jointly enforcing intra-class compactness and inter-class separability in the latent space to co-optimize representations of both old and new class prototypes. Evaluated on CIFAR-10, CIFAR-100, and a GNSS interference classification dataset, the approach consistently outperforms state-of-the-art methods, achieving simultaneous improvements in both final accuracy and forgetting rate. These results demonstrate its effectiveness in enhancing representation robustness and sustaining discriminative capability across incremental tasks.

Hard negative sampling in contrastive learning critically influences representation geometry, yet its precise roles in mitigating dimensional collapse (DC) and inducing neural collapse (NC) remain poorly understood. Method: We develop a generalized contrastive loss framework, integrating equiangular tight frame (ETF) geometric modeling and unit-sphere normalization analysis. Contribution/Results: We provide the first rigorous proof that, under both supervised and unsupervised hard contrastive learning (HSCL/HUCL), any global optimum necessarily exhibits NC—i.e., class means form an ETF and intra-class features collapse to identical points. Crucially, we extend this result to generic losses including InfoNCE without assuming class-conditional independence. Theory and experiments jointly demonstrate that NC emerges stably *only* when hard negative sampling is synergistically combined with feature normalization under Adam-based batch optimization; otherwise, DC prevails. Our code is publicly available.

Generalized Category Discovery (GCD) aims to jointly cluster unlabeled data—containing both known and novel classes—by leveraging labeled data from known classes; its core challenge lies in accuracy imbalance caused by distributional ambiguity between known and novel classes. This paper proposes a Joint Prototype Learning (JPL) framework: (1) unifying prototype modeling for both known and novel classes to eliminate classifier bias; (2) introducing a two-level adaptive pseudo-labeling mechanism to mitigate confirmation bias; and (3) integrating contrastive regularization and clustering consistency constraints to align feature representations with clustering objectives, further enhanced by novel-class cardinality estimation and outlier detection for task-level co-optimization. Evaluated on both generic and fine-grained benchmarks, JPL achieves state-of-the-art performance, significantly improving balanced accuracy across known and novel classes while enhancing representation discriminability.

This work addresses the challenge of learning normalized embedding representations that are both intra-class compact and inter-class angularly separable while preserving neural network expressiveness and accelerating convergence. To this end, the authors propose the CoCo loss function, which uniquely unifies intra-class representation collapse and inter-class contrast within a single objective. Operating in the normalized embedding space, CoCo guides the network toward a geometrically optimal configuration, offering both optimization flexibility and strong clustering dynamics. Theoretical analysis and gradient characterization demonstrate its compatibility with diverse architectures. Empirical evaluation shows that CoCo matches or surpasses kernel SVMs, random forests, and cross-entropy baselines across multiple tabular datasets from OpenML-CC18, while significantly improving convergence speed and intra-class compactness.

Dimensional collapse in the embedding space remains a critical challenge in semi-supervised contrastive learning, degrading representation discriminability. Method: We propose CLOP (Contrastive Learning with Orthogonal Prototypes), a novel loss function that geometrically regularizes the embedding structure by enforcing class prototypes to span orthogonal linear subspaces—thereby fundamentally mitigating dimensional collapse. We first identify and quantify the critical learning rate threshold at which standard contrastive loss induces collapse, and leverage this insight to design an orthogonal-prototype-driven semi-supervised objective. CLOP integrates contrastive learning, orthogonal constraint optimization, and geometric embedding-space regularization. Results: CLOP achieves significant performance gains on both image classification and object detection benchmarks. Crucially, it exhibits strong robustness to variations in learning rate and batch size—addressing key practical limitations of existing contrastive methods.

This study investigates whether training biases can drive interpretable specialization of hidden neurons in minimal Gaussian-activated MLPs and enhance the ability to reconstruct training data from weights. By introducing three structured regularization losses—coverage, separation, and response overlap—in networks whose width equals the dataset size, the work systematically evaluates their impact on neuronal specialization and prototype reconstruction. Experiments demonstrate that coverage regularization significantly increases prototype utilization and reduces reconstruction error, whereas purely repulsive losses, without compatible attractive terms, cause prototypes to collapse outside the convex hull of the input data. Based on 480 controlled experiments (N=3–100), the study validates the design principle that “repulsion must be paired with attraction,” offering an effective mechanism for achieving interpretable neuron specialization.

This work investigates under what positive sample sampling conditions contrastive learning can recover a meaningful geometric structure in the latent space. By constructing a measure-theoretic framework, the study introduces a “diversity condition” as a necessary requirement for the identifiability of latent geometry and elucidates the joint influence of sampling support and encoder inductive bias on representation identifiability. Theoretically, it is shown that under full-support sampling, the global optimum of InfoNCE recovers the latent structure up to orthogonal equivalence; however, under non-full support, non-orthogonal mappings may yield better solutions. To address this, the authors propose a support-corrected variant of InfoNCE and model representations using the von Mises–Fisher distribution, empirically validating on both synthetic and real-world data the critical role of inductive bias when sampling diversity is limited.

This work addresses the degradation of embedding space coherence and model performance in federated learning caused by premature alignment of immature prototypes, particularly under highly non-IID data distributions. To mitigate this issue, the authors propose FedSAP, a novel framework that integrates a delayed-alignment curriculum scheduling mechanism with a geometry-driven proxy separation loss on the unit hypersphere. This approach enhances intra-class compactness and inter-class separability without increasing communication overhead. As the first study to formalize scheduled alignment as a general design principle in federated prototype learning, FedSAP stabilizes representation learning without introducing additional parameters and naturally extends to semi-supervised settings. Extensive experiments demonstrate consistent improvements, with up to a 4-percentage-point gain over state-of-the-art methods across three benchmark datasets, especially excelling in high-heterogeneity scenarios.