heterogeneity assessment

Data heterogeneity severely degrades model robustness, out-of-distribution (OOD) generalization, and fairness—challenges inadequately addressed by conventional average-performance optimization. This paper proposes a full-stack heterogeneity-aware learning framework that, for the first time, explicitly models data heterogeneity as a core, quantifiable, diagnosable, and intervenable dimension across the entire ML lifecycle: data acquisition, modeling, evaluation, and deployment. Our approach integrates hierarchical heterogeneity measurement, domain-adaptive regularization, counterfactual fairness constraints, multi-granularity evaluation protocols, and diagnosis-driven iterative optimization. Extensive validation across healthcare and finance domains demonstrates substantial improvements: +18.7% OOD accuracy across domains, −42% reduction in demographic parity gap (ΔDP), and significantly enhanced decision interpretability. The framework provides both theoretical foundations and practical guidelines for building trustworthy AI systems.

This study addresses the distortion of futility conclusions in clinical trial interim analyses caused by deviation of the enrolled population from the target population. We propose a robust futility stopping rule that innovatively integrates permutation-based variable screening to identify sources of heterogeneity, coupled with a post-hoc hybrid adjustment strategy combining model-based prediction and conventional stratification—leveraging all baseline covariates for comprehensive calibration. Through systematic simulation, we evaluate how subgroup imbalance affects various stratification approaches (naïve, model-driven, and hybrid). Results demonstrate that our hybrid strategy effectively corrects for interim population drift, substantially improving the accuracy of futility decisions, statistical power, and decision completeness. This approach provides a more reliable evidentiary foundation for early stopping in adaptive trial designs.

Clinical oncology trials frequently yield false-negative results due to treatment effect heterogeneity across patient subgroups—especially when subgroup analyses are not prespecified in the trial design. To address this, we propose a novel post-hoc subgroup efficacy testing framework that leverages external data (e.g., published clinical studies and electronic health records) to enhance statistical power. Our method employs a permutation-based inference procedure that requires no strong modeling assumptions, rigorously controls Type I error under arbitrary unmeasured confounding and population distribution shift, and achieves theoretical optimality in power. Evaluated on a multicenter retrospective glioblastoma study and extensive simulations, the approach significantly improves detection sensitivity for heterogeneous treatment effects, thereby mitigating decision distortion caused by effect heterogeneity. It provides a generalizable, statistically principled tool for re-evaluating trial null findings.

This work addresses combinatorial bias in classification evaluation metrics under few-shot settings, where group-size disparities induce spurious fairness assessments. Through probabilistic modeling and combinatorial analysis, we systematically identify and quantify the latent bias mechanisms inherent in common metrics—including accuracy and F1-score—under sample-size imbalance. We propose a model-agnostic framework for detecting and correcting such bias, unifying treatment of undefined cases (e.g., zero-denominator scenarios) and metric sensitivity to class distribution. Our approach substantially enhances discriminative power in fairness evaluation under data scarcity, mitigating erroneous attribution and misguided policy interventions arising from metric distortion. The framework provides both theoretical grounding and practical tools for trustworthy AI assessment in resource-constrained environments.

This paper addresses a fundamental question in causal mechanism identification: under what conditions can heterogeneous treatment effects (HTEs) be used to infer the activation of a specific causal mechanism? It highlights that prevailing HTE detection methods—relying on pre-treatment covariates—implicitly assume linearity or additivity, rendering them invalid for mechanism inference under nonlinear outcome generation. Method: We formally characterize the necessary and sufficient conditions under which HTEs carry identifying information about mechanism activation. Using potential outcomes frameworks, mechanism identification theory, and HTE modeling, we prove that nonlinear transformations of outcomes generally eliminate the inferential value of HTEs for mechanisms. Contribution: We derive testable experimental design principles and an interpretive framework that explicitly delineate the boundaries of mechanism identification. Our results provide empiricists with robust identification guidelines and a systematic pathway for sensitivity analysis, bridging theoretical causality and applied HTE estimation.

This study addresses the critical gap in understanding how label bias and selection bias systematically affect the evaluation, performance, and efficacy of fairness mitigation methods in classification models, often leading to unrepresentative assessments. To this end, we propose the first evaluation framework that enables controlled introduction of distinct bias types. By constructing a “fair world” from a real low-discrimination dataset and its biased variants, our approach disentangles the individual effects of label and selection bias. Evaluating models and fairness interventions on an unbiased test set reveals that the type of bias significantly modulates the effectiveness of mitigation strategies. Notably, under unbiased conditions, we find no inherent trade-offs between fairness and accuracy or between individual and group fairness.

Clinical prediction models often suffer from label bias due to disparities in diagnostic frequencies across subpopulations—such as those defined by sex, race, or diabetes status—leading to systematic prediction errors and distorted performance evaluation. This work proposes the first framework that integrates causal inference with a hidden Markov model to define a counterfactual target: the probability that an individual would be diagnosed under the reference group’s diagnostic rate. By explicitly modeling the latent disease progression process and observed testing outcomes, the method corrects for biases arising from differential diagnostic delays. In simulations, it reduces the Observed:Expected ratio for the previously underestimated group from 1.34 to 1.02. Applied to real-world chronic kidney disease data, it improves calibration for non-diabetic patients, lowering their ratio from 1.55 to 1.01.

Existing methods for detecting risk heterogeneity in population-imbalanced biomedical data are often compromised by misspecification of baseline models and regularization bias, leading to unstable inference. This work proposes a robust semiparametric inference framework grounded in Neyman orthogonality, which mitigates the influence of nuisance parameter estimation errors through orthogonalization, thereby enabling more accurate and stable identification of heterogeneity in finite samples. The proposed estimator enjoys favorable theoretical properties, including local robustness and asymptotic normality. In both simulation studies and real-world analysis of eICU data, the method successfully uncovers ethnicity-specific diagnostic risk disparities that standard likelihood-based approaches fail to detect, demonstrating substantial clinical relevance.

This study addresses significant population biases in biomedical AI that originate as early as the data generation and study design phases, particularly due to missing or imbalanced ancestral information in omics datasets, thereby exacerbating health disparities. By systematically analyzing 4,719 PubMed-indexed omics papers from 2015 to 2024 and major repositories such as CellxGene and GEO—integrating automated literature mining, demographic analysis, and modeling of bias propagation in foundation models—the work traces the roots of health inequities to the pre-training data ecosystem. It reveals that ancestral metadata are rarely reported and that existing omics data are overwhelmingly skewed toward European ancestry. To mitigate these issues, the study proposes a systematic framework grounded in three principles: provenance awareness, openness, and transparent evaluation, aiming to foster more equitable and robust biomedical AI from the research outset.

This study addresses the systemic measurement bias in pulse oximeters across racial groups, which leads to inequitable health decisions. To tackle this issue, the authors propose a data fairness analysis framework that explicitly decomposes fairness into three actionable dimensions: data, prediction, and decision. Leveraging oracle-based causal simulation and counterfactual attribution analysis, they develop a statistical provenance model that traces how upstream informational biases propagate and amplify into clinical disparities and adverse health outcomes. By translating abstract notions of fairness into empirically testable statistical metrics, the framework establishes a reproducible analytical paradigm for identifying and mitigating such systemic inequities, thereby underscoring the pivotal role of statistics in AI-driven healthcare.