AdaPRL: Adaptive Pairwise Regression Learning with Uncertainty Estimation for Universal Regression Tasks

2025-01-10

AI Summary

Existing deep regression models neglect inter-sample ordinal relationships and struggle to model heteroscedastic uncertainty, resulting in limited generalization, poor noise robustness, and weak performance under small-sample regimes. To address these limitations, we propose AdaPRL, an Adaptive Pairwise Regression Learning framework—the first to jointly integrate pairwise ordinal learning with deep probabilistic modeling. AdaPRL simultaneously captures relative sample ordering and input-dependent heteroscedastic uncertainty via four core components: dynamic weight adaptation, uncertainty-aware probabilistic regression, multi-task shared representation learning, and a tailored pairwise loss. Evaluated across over ten real-world regression tasks—including recommendation, age estimation, finance, and natural language understanding—AdaPRL consistently outperforms state-of-the-art methods: achieving an average 2.3% improvement in prediction accuracy, a 37% gain in noise robustness, and markedly enhanced small-sample robustness. Moreover, it supports multivariate time-series forecasting and enables interpretable analysis.

Technology Category

Reasoning under Uncertainty: Relational Probabilistic ModelsMachine Learning: Calibration & Uncertainty QuantificationGame Theory and Economic Paradigms: Adversarial Learning

Application Category

Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for rankingUser Modeling, Personalization and Recommendation: Fairness-aware retrieval and rankingGraph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs

Abstract

Current deep regression models usually learn in point-wise way that treat each sample as an independent input, neglecting the relative ordering among different data. Consequently, the regression model could neglect the data 's interrelationships, potentially resulting in suboptimal performance. Moreover, the existence of aleatoric uncertainty in the training data may drive the model to capture non-generalizable patterns, contributing to increased overfitting. To address these issues, we propose a novel adaptive pairwise learning framework (AdaPRL) for regression tasks which leverages the relative differences between data points and integrates with deep probabilistic models to quantify the uncertainty associated with the predictions. Additionally, we adapt AdaPRL for applications in multi-task learning and multivariate time series forecasting. Extensive experiments with several real-world regression datasets including recommendation systems, age estimation, time series forecasting, natural language understanding, finance, and industry datasets show that AdaPRL is compatible with different backbone networks in various tasks and achieves state-of-the-art performance on the vast majority of tasks, highlighting its notable potential including enhancing prediction accuracy and ranking ability, increasing generalization capability, improving robustness to noisy data, improving resilience to reduced data, and enhancing interpretability, etc.

Problem

• Deep Learning
• Regression Analysis
• Uncertainty Estimation

Innovation

• AdaPRL
• Relative Difference Learning
• Probabilistic Model Uncertainty