The Multiclass Score-Oriented Loss (MultiSOL) on the Simplex

2025-11-27

AI Summary

In multi-class classification, optimizing evaluation metrics (e.g., F1, Kappa) typically relies on post-hoc threshold tuning—a manual, non-differentiable process that hinders end-to-end optimization and struggles with class imbalance. Method: We propose MultiSOL, the first score-oriented loss function generalized from binary to multi-class settings. It models multi-dimensional decision thresholds as a prior distribution over the probability simplex and leverages the simplex’s geometric structure to construct a fully differentiable, metric-driven loss. This enables direct, end-to-end optimization of target metrics during training. Contribution/Results: MultiSOL establishes a unified, metric-driven optimization framework that obviates heuristic threshold tuning. Experiments across multiple benchmark datasets show that MultiSOL matches or exceeds state-of-the-art losses in performance while demonstrating superior stability across diverse metrics and robust generalization under varying class distributions, especially in imbalanced scenarios.

Technology Category

• Machine Learning: Multi-class/Multi-label Learning & Extreme Classification
• Search and Optimization: Learning to Search
• Computer Vision: Learning & Optimization for CV

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

In the supervised binary classification setting, score-oriented losses have been introduced with the aim of optimizing a chosen performance metric directly during the training phase, thus avoiding extit{a posteriori} threshold tuning. To do this, in their construction, the decision threshold is treated as a random variable provided with a certain extit{a priori} distribution. In this paper, we use a recently introduced multidimensional threshold-based classification framework to extend such score-oriented losses to multiclass classification, defining the Multiclass Score-Oriented Loss (MultiSOL) functions. As also demonstrated by several classification experiments, this proposed family of losses is designed to preserve the main advantages observed in the binary setting, such as the direct optimization of the target metric and the robustness to class imbalance, achieving performance comparable to other state-of-the-art loss functions and providing new insights into the interaction between simplex geometry and score-oriented learning.

Problem

• Extends score-oriented losses to multiclass classification using MultiSOL.
• Directly optimizes target metrics without threshold tuning post-training.
• Enhances robustness to class imbalance via simplex geometry insights.

Innovation

• Extends score-oriented losses to multiclass classification
• Treats decision thresholds as random variables with priors
• Directly optimizes target metrics without threshold tuning