Robust Losses from Univariate Base Functions for Noisy-Label Learning

2026-07-18

AI Summary

This work addresses the instability of deep neural networks when trained under label noise by proposing a general framework that systematically constructs robust multiclass loss functions from univariate base functions for the first time. By introducing a mapping operator, the approach explicitly links the properties of base functions to loss robustness and develops two complementary construction mechanisms—target separation and binary reduction. Theoretical analysis reveals critical conditions governing the symmetry and asymmetry of loss functions, establishing new principles and design pathways for noise-robust learning. Extensive experiments demonstrate that the proposed method achieves state-of-the-art or superior performance across various synthetic and real-world noisy datasets.

Abstract

Learning with noisy labels is a fundamental problem in training reliable deep neural networks. Robust loss functions provide a direct and effective way to mitigate the adverse effects of label noise. However, most existing robust losses are designed directly at the level of the final multiclass objective, which makes it difficult to systematically characterize and extend their robustness properties. In this paper, we propose a general framework that constructs robust multiclass losses from univariate base functions. By defining mapping operators from base functions to multiclass losses, the robustness of the induced losses can be characterized through simple properties of the base functions. We develop two complementary construction schemes, Target Separation and Binary Reduction, corresponding to inter-class independent and inter-class dependent formulations, respectively. For both schemes, we analyze their symmetry and asymmetry properties and derive corresponding sufficient conditions, which provide theoretical criteria for noise-robust loss design. The proposed framework also provides a new route to constructing symmetric losses, serving as a complement to normalization-based symmetric loss designs. Extensive experiments on synthetic and real-world noisy-label benchmarks demonstrate that the proposed losses achieve competitive or superior performance under various noise settings.

Problem

• noisy-label learning
• robust loss
• label noise
• multiclass classification
• deep neural networks

Innovation

• robust loss
• noisy-label learning
• univariate base function
• symmetry analysis
• multiclass loss construction