🤖 AI Summary
This work addresses the challenge of learning normalized embedding representations that are both intra-class compact and inter-class angularly separable while preserving neural network expressiveness and accelerating convergence. To this end, the authors propose the CoCo loss function, which uniquely unifies intra-class representation collapse and inter-class contrast within a single objective. Operating in the normalized embedding space, CoCo guides the network toward a geometrically optimal configuration, offering both optimization flexibility and strong clustering dynamics. Theoretical analysis and gradient characterization demonstrate its compatibility with diverse architectures. Empirical evaluation shows that CoCo matches or surpasses kernel SVMs, random forests, and cross-entropy baselines across multiple tabular datasets from OpenML-CC18, while significantly improving convergence speed and intra-class compactness.
📝 Abstract
In this work, we introduce CoCo, a loss function aimed at learning normalized and well-structured representations. The proposed loss encourages intra-class collapse and inter-class contrast while preserving sufficient flexibility for neural networks to approximate geometrically optimal embeddings with large angular separation between classes. We provide a theoretical analysis positioning CoCo with respect to related objectives such as dot regression and cross-entropy, showing that the new proposed loss benefits from closer initialization to the optimal configuration, more informative gradients, and stronger incentives for class-wise representation collapse. Extensive experiments on diverse tabular datasets from the OpenML-CC18 benchmark show that CoCo achieves competitive performance with state-of-the-art methods, including kernel SVM, Random Forest, dot regression, and cross-entropy-based neural networks. In addition, both theoretical arguments and empirical analyses demonstrate that the proposal promotes tighter class clustering and faster convergence. These results highlight CoCo loss as an effective objective for learning discriminative representations while maintaining competitive predictive performance.