balanced accuracy computation

This paper addresses data imbalance in regression tasks—where target variables are continuous—a problem extensively studied in classification but lacking systematic investigation in regression. We propose the first taxonomy of resampling methods specifically designed for imbalanced regression. Our framework systematically evaluates oversampling, undersampling, and hybrid strategies across three dimensions: regression models (linear regression, tree-based models, neural networks), learning processes, and specialized evaluation metrics (uM, wMAE). Experimental results demonstrate that judicious resampling significantly improves predictive accuracy in sparse regions of the target space, and that model sensitivity to sampling strategies varies substantially. We uncover mechanistic insights into how resampling operates effectively in continuous output spaces. To foster reproducibility and further research, we publicly release all source code and benchmark datasets. This work establishes a rigorous, extensible analytical framework and practical guidelines for addressing imbalance in regression.

Existing classification performance metrics suffer from inconsistent scales and high sensitivity to class imbalance, rendering cross-dataset evaluation incomparable and difficult to interpret. To address this, we propose the Outperformance Score (OS), a unified normalization framework that maps any confusion-matrix-based metric onto the [0,1] interval. OS is defined as the percentile rank of an observed performance value within a reference empirical distribution induced by the class imbalance ratio. This constitutes the first general-purpose standardization framework that endows diverse CMBCP metrics with consistent semantics and comparable scales. Crucially, OS eliminates reliance on fixed decision thresholds or parametric distributional assumptions, enabling robust evaluation under dynamically varying imbalance ratios. Extensive validation across real-world datasets from healthcare, finance, and natural language processing domains demonstrates that OS significantly enhances reliability and consistency in both inter-metric and cross-dataset comparisons, while supporting plug-and-play integration.

This paper challenges the widely held assumption in machine learning that the Area Under the Precision-Recall Curve (AUPRC) is universally superior to the Area Under the ROC Curve (AUROC) under class imbalance—and investigates its implications for algorithmic fairness. Method: We conduct rigorous theoretical analysis, experiments on semi-synthetic and real-world fairness-sensitive datasets, and a large-scale bibliometric study covering over one million publications. Contribution/Results: We provide the first formal proof that AUPRC is not generally advantageous under extreme imbalance; instead, it systematically amplifies group-level bias by favoring subpopulations with higher positive-class density. We trace and empirically refute the long-standing misconception that “AUPRC is inherently better.” Furthermore, we establish verifiable criteria delineating the applicability boundaries of AUROC versus AUPRC, and propose a principled, imbalance- and subgroup-aware framework for metric selection—thereby offering both theoretical foundations and practical guidance for fair model evaluation.

Existing binary classifier evaluation lacks a consensus standard, and common metrics are frequently misapplied in class-imbalanced and cost-sensitive settings. Method: We propose Weighted Accuracy (WA) as a unified evaluation metric strictly aligned with minimization of Total Classification Cost (TCC). Grounded in sample-level cost modeling, we develop a scalable, cost-sensitive evaluation framework applicable to any metric expressible as a linear combination of sample-dependent quantities. Contribution/Results: We formally characterize the limited validity of rebalancing techniques—showing they are only optimal under specific cost structures—and introduce a WA estimation method that requires no complete prior knowledge of misclassification costs. Extensive experiments across diverse datasets, distribution shifts, and deployment scenarios demonstrate WA’s high correlation with and robustness to TCC, significantly enhancing evaluation comparability and practical utility.

Class imbalance severely degrades anomaly detection performance, yet its theoretical underpinnings remain poorly understood. Method: We establish a rigorous teacher–student perceptron framework and, for the first time, apply replica theory to derive exact analytical solutions under arbitrary class imbalance. Contribution/Results: We systematically disentangle three distinct imbalance sources—intrinsic, training-set, and test-set—and prove that the optimal training imbalance ratio is generally not 50%, but instead evolves nontrivially with intrinsic imbalance, sample size, and noise level. We identify a sharp performance crossover between low-noise and high-noise regimes. Our theory quantitatively characterizes the individual impact of each imbalance source and yields actionable guidelines for constructing training sets—directly challenging the empirical “balance-is-optimal” heuristic. This work provides the first unified theoretical foundation for imbalanced learning in binary classification.

Linear classifiers (e.g., SVM) suffer from degraded generalization performance on high-dimensional imbalanced data. Method: We establish a high-dimensional asymptotic theoretical framework and, for the first time, rigorously derive analytical expressions for the generalization error under undersampling and oversampling. Our approach integrates random matrix theory, high-dimensional statistical learning, and unsupervised probabilistic modeling–driven resampling. Contribution/Results: We quantify how resampling efficacy depends on the first- and second-order statistics of the data and the choice of evaluation metric. Crucially, we prove—and empirically verify—that hybrid sampling consistently outperforms either undersampling or oversampling alone. Extensive numerical experiments and evaluations on real-world datasets—including deep neural network features—demonstrate strong agreement between theoretical predictions and empirical results, with substantial improvements in minority-class classification accuracy. This work provides an interpretable, generalizable, and principle-based foundation for data rebalancing in high dimensions.

This study addresses the challenge posed by severe class imbalance in anomaly detection, which complicates the interpretation and comparison of common evaluation metrics. The authors systematically analyze the behavior of AUROC, AUPR, F1-score, and Matthews Correlation Coefficient (MCC) across varying anomaly ratios and introduce a novel "metric landscape" visualization technique. This approach reveals, for the first time, each metric’s inherent preference for true positive rate versus true negative rate and how their stability varies with imbalance levels. By modeling the relationship between metrics and anomaly prevalence, the work delineates clear applicability boundaries for each metric, thereby providing a principled, interpretable foundation for reliable metric selection in highly imbalanced anomaly detection scenarios.

Class imbalance significantly degrades classification performance, yet a unified theoretical framework has been lacking. This work proposes the first model-agnostic, geometrically interpretable three-parameter theory—characterized by the imbalance ratio η, dimensionality ratio κ, and class separability Δ—to analytically derive the closed-form Bayes error under a Gaussian generative model. The framework elucidates the mechanism of decision boundary shift and systematically identifies four distinct performance degradation regimes. Theoretical predictions align closely with empirical results on high-dimensional genomic data: when log(η) > Δ√κ, minority-class recall drops sharply while precision rises asymmetrically, and both F1 score and PR-AUC degrade in accordance with the proposed mechanism, thereby validating the framework’s predictive power and broad applicability.

This study addresses a critical limitation in existing credit fraud detection models, which typically optimize for transaction count rather than actual financial loss. Under controlled class imbalance, the work systematically disentangles and evaluates the distinct roles of monetary amount—first as sample weights during training and second as a reranking variable post-prediction. Building upon XGBoost, the approach integrates amount-derived features (e.g., ratios, velocities), multiple weighting strategies, and a score×amount reranking mechanism, with performance assessed using dollar-based metrics such as dollar recall and dollar precision. Empirical results demonstrate that incorporating amount-aware features and reranking substantially improves dollar recall, yet aggressive amount-based weighting degrades ranking quality. The findings indicate that transaction amount alone is unsuitable as a standalone sample weighting rule.

Whether class imbalance correction improves the performance of clinical prediction models remains controversial. This study leverages data from the GUSTO-I clinical trial to systematically evaluate the impact of various correction strategies—including algorithm-level rebalancing, oversampling, and hybrid sampling—on model discrimination (AUC), calibration (calibration plots and MAPE), and predictive stability (Classification Instability Index, CII) across varying sample sizes. Using penalized logistic regression with 200 bootstrap replications, we find that all correction methods fail to enhance discriminative performance and instead introduce greater calibration bias, risk overestimation, and increased prediction instability. These results challenge the common practice of routinely applying class imbalance corrections in clinical modeling and, for the first time in large-scale simulations, reveal their potential harms.

This study investigates the mechanism by which synthetic data augmentation improves score-based classification performance—measured by metrics such as AUROC and AUPRC—in class-imbalanced settings. By developing a theoretical framework that disentangles the effects of augmentation on effective class weighting and distributional bias, and integrating tools from statistical learning theory, minimax analysis, and finite-sample error decomposition, the work establishes that under correctly specified models, augmentation solely reduces variance without improving overall performance. However, under model misspecification, it can mitigate ranking errors by correcting class imbalance. The analysis yields novel minimax lower bounds, which are corroborated through simulation experiments.