Population-Level Decision Curve Analysis May Mislead the Evaluation of Prediction Model Usefulness under Subgroup Utility Heterogeneity

2026-09-28

AI Summary

This study addresses the risk that subgroup utility heterogeneity may lead decision curve analysis (DCA) at the population level to misjudge the true clinical utility of predictive models. Through rigorous mathematical derivation, this work constructs a discordance region model to quantify, for the first time, the interference boundaries imposed by such heterogeneity on DCA conclusions, validated via subgroup-specific net benefit analyses and real-world case studies. The findings reveal that when incremental net benefits across subgroups exhibit opposing signs, population-level net benefit may obscure true utility. Accordingly, an early-warning mechanism based on sign-discordant ΔNB and a robustness evaluation framework are proposed. This research demonstrates that employing population net benefit as a proxy for utility necessitates the assumption of subgroup homogeneity, thereby providing practical tools for identifying and mitigating failures in DCA-based conclusions.

Abstract

Background Decision curve analysis (DCA) evaluates prediction model usefulness using net benefit (NB). Population-level NB is often interpreted as a proxy for population-level expected utility, but this assumes comparability of utility across subgroups. We aimed to characterize when subgroup utility heterogeneity may invalidate population-level DCA conclusions. Methods We compared prediction-driven treatment with default strategies in populations containing subgroups with different utility values. We derived an inconsistency region: combinations of subgroup-specific ΔNB values for which population-level NB and utility favor different strategies. We also developed a practical robustness framework. Results Opposite signs of subgroup-specific ΔNB provide a warning signal for possible inconsistency. The inconsistency region is larger when subgroup sizes are more similar and subgroup-specific \(a-c\) values are more different, where \(a-c\) is the incremental utility of a true positive relative to a false negative. When \(a-c\) differs across subgroups, population-level NB combines quantities on different implicit utility scales and may conflict with population utility. A real-world case study illustrates the problem. Conclusions Using population-level NB as a proxy for population utility implicitly assumes homogeneous \(a-c\) across subgroups. Subgroup DCA and our framework can identify and assess when utility heterogeneity may invalidate population-level conclusions.

Problem

• Decision Curve Analysis
• Net Benefit
• Subgroup Utility Heterogeneity
• Prediction Model Evaluation
• Expected Utility

Innovation

• Decision Curve Analysis
• Subgroup Utility Heterogeneity
• Net Benefit
• Inconsistency Region
• Robustness Framework