Predictiveness Curve Assessment under Competing Risks for Risk Prediction Models

2025-07-31

AI Summary

To address the challenge of evaluating risk prediction models under competing risks, this paper introduces a novel Predictiveness Curve method that characterizes the relationship between model-predicted quantiles and cumulative incidence probabilities, enabling risk stratification and clinical threshold selection. It is the first work to extend the Predictiveness Curve framework to competing risks settings. The proposed approach integrates cross-validation, flexible regression modeling (e.g., spline-based regression), and perturbation resampling to yield unbiased estimation and valid statistical inference for τ-year event risk. Simulation studies demonstrate excellent small-sample performance and robustness. The method is applied to assess a risk prediction model for liver disease–related mortality in a cirrhosis cohort, clearly revealing the distributional characteristics of predictive capacity. An accompanying open-source R package is provided.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Relational Probabilistic Models
• Planning, Routing, and Scheduling: Scheduling under Uncertainty

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• User Modeling, Personalization and Recommendation: User model development and evaluation
• Graph Algorithms and Modeling for the Web: Querying, indexing, and retrieval in Web-related graphs

Abstract

The predictiveness curve is a valuable tool for predictive evaluation, risk stratification, and threshold selection in a target population, given a single biomarker or a prediction model. In the presence of competing risks, regression models are often used to generate predictive risk scores or probabilistic predictions targeting the cumulative incidence function--distinct from the cumulative distribution function used in conventional predictiveness curve analyses. We propose estimation and inference procedures for the predictiveness curve with a competing risks regression model, to display the relationship between the cumulative incidence probability and the quantiles of model-based predictions. The estimation procedure combines cross-validation with a flexible regression model for tau-year event risk given the model-based risk score, with corresponding inference procedures via perturbation resampling. The proposed methods perform satisfactorily in simulation studies and are implemented through an R package. We apply the proposed methods to a cirrhosis study to depict the predictiveness curve with model-based predictions for liver-related mortality.

Problem

• Assessing predictiveness curves under competing risks for risk prediction models
• Estimating cumulative incidence probability versus model-based prediction quantiles
• Developing methods for flexible regression and inference in competing risks scenarios

Innovation

• Competing risks regression model for predictiveness curve
• Cross-validation with flexible regression model
• Perturbation resampling for inference procedures