🤖 AI Summary
This work addresses the challenges in ordinal regression posed by ambiguous label boundaries and noise arising from subjective annotations, as well as the bias amplification inherent in conventional methods that rely on fixed supervision targets. To overcome these limitations, the authors propose the D3O framework, which replaces static supervision with dynamic distribution distillation. D3O leverages a self-distillation mechanism to evolve ordinal label distributions and incorporates a contrastive ordinal-aware label augmentation module alongside a cumulative distribution function (CDF)-based cross-layer interactive distillation strategy. This approach effectively models inter-class ambiguity and instance-level uncertainty while integrating vision-language alignment to enrich label representation. Evaluated across four ordinal regression benchmarks, D3O consistently outperforms state-of-the-art methods, demonstrating exceptional robustness under severe class imbalance and highly noisy supervision.
📝 Abstract
Ordinal regression is widely used in scenarios where labels are discrete yet inherently ordered. In practice, however, ordinal labels are often obtained by discretizing underlying continuous semantics through subjective human judgment, resulting in ambiguous boundaries and annotation noise. Such uncertainty challenges existing methods that rely on fixed supervision targets, which may reinforce biased ordering under subjective annotations. To address this limitation, we propose D3O, a dynamic distribution distillation framework that replaces static supervision with training-driven evolution of ordinal label distributions via self-distillation. Specifically, we introduce a contrastive ordinal-aware label enhancement module that leverages vision-language alignment to recover refined label distributions capturing both inter-class ambiguity and instance-level uncertainty. Furthermore, we design a CDF-based cross-layer interaction distillation mechanism to propagate cumulative ordinal structure across network hierarchy, ensuring consistent ordinal geometry in intermediate representations. Extensive experiments on four general ordinal regression tasks demonstrate that our proposed D3O consistently outperforms existing approaches, particularly under severe class imbalance and noisy supervision. These results highlight the effectiveness of dynamic supervision in learning robust ordinal representations beyond fixed targets. The code will be publicly available.