Balanced Sharpness-Aware Minimization for Imbalanced Regression

2025-08-23

AI Summary

In visual regression, model generalization suffers significantly on rare target values due to imbalanced target-value distributions. This paper reformulates imbalanced regression as an “imbalanced generalization” problem and proposes Balanced Sharpness-Aware Minimization (BSAM). BSAM is the first method to address this issue from the perspective of loss sharpness: it introduces a targeted reweighting strategy in the observation space, jointly optimizing generalization across target-value regions via gradient perturbation analysis and dynamic sample weighting. We theoretically establish that BSAM yields a tighter generalization error bound. Extensive experiments on age estimation and depth estimation demonstrate that BSAM consistently outperforms state-of-the-art methods, substantially mitigating performance bias toward frequent targets while enhancing model robustness and overall generalization capability.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Computer Vision: Learning & Optimization for CV
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

Regression is fundamental in computer vision and is widely used in various tasks including age estimation, depth estimation, target localization, etc However, real-world data often exhibits imbalanced distribution, making regression models perform poorly especially for target values with rare observations~(known as the imbalanced regression problem). In this paper, we reframe imbalanced regression as an imbalanced generalization problem. To tackle that, we look into the loss sharpness property for measuring the generalization ability of regression models in the observation space. Namely, given a certain perturbation on the model parameters, we check how model performance changes according to the loss values of different target observations. We propose a simple yet effective approach called Balanced Sharpness-Aware Minimization~(BSAM) to enforce the uniform generalization ability of regression models for the entire observation space. In particular, we start from the traditional sharpness-aware minimization and then introduce a novel targeted reweighting strategy to homogenize the generalization ability across the observation space, which guarantees a theoretical generalization bound. Extensive experiments on multiple vision regression tasks, including age and depth estimation, demonstrate that our BSAM method consistently outperforms existing approaches. The code is available href{https://github.com/manmanjun/BSAM_for_Imbalanced_Regression}{here}.

Problem

• Addresses imbalanced regression in computer vision tasks
• Measures generalization via loss sharpness in observation space
• Proposes balanced sharpness-aware minimization for uniform generalization

Innovation

• Balanced Sharpness-Aware Minimization for generalization
• Targeted reweighting strategy homogenizes generalization ability
• Perturbation-based loss sharpness measurement in observation space