A Statistical Framework for Data-Driven Discovery of Differential Performance in Clinical Risk Prediction Models

2026-08-02

AI Summary

Clinical risk prediction models may exhibit performance disparities across multidimensional subgroups not predefined during development, leading to unfair clinical decisions. Existing fairness evaluation methods rely on manually specified subgroups and struggle to uncover vulnerable populations defined by higher-order feature interactions. This work proposes the “unfairness tree” (utree)—a data-driven recursive partitioning framework that automatically identifies and quantifies model performance heterogeneity across complex feature combinations without requiring prespecified subgroups. Through simulation studies and empirical analyses across multiple models on the GUSTO-I acute myocardial infarction dataset, utree effectively detects performance gaps associated with age, sex, blood pressure, and Killip class while controlling Type I error, thereby overcoming key limitations of conventional fairness assessment approaches.

Abstract

Predictive models employing artificial intelligence (AI) and machine learning (ML) are increasingly being used for decision support in healthcare settings. These models may exhibit differential performance across population subgroups defined by race, age, sex, and other factors and cause disparate clinical impacts, leading to intensive recent study of what has been termed "model fairness". While many methods have been proposed to assess risk prediction model fairness, these techniques generally require that the end user pre-specify the groups across which fairness is to be evaluated. In real-world settings, however, important model performance disparities may arise in unknown subgroups defined by multiple intersecting characteristics. To address this problem, we propose the unfairness tree (utree), a data-driven recursive partitioning framework for identifying subgroups with differential model performance. In simulations, the utree exhibits nominal empirical type I error rates and good ability to detect, quantify, and characterize performance discrepancies defined by higher-order variable interactions. In six mortality risk models fit to the GUSTO-I acute myocardial infarction trial dataset, utrees identified subgroup-specific performance patterns, with age, sex, blood pressure, and Killip class consistently associated with differential model performance.

Problem

• model fairness
• differential performance
• subgroup identification
• clinical risk prediction
• healthcare disparities

Innovation

• unfairness tree
• recursive partitioning
• model fairness
• differential performance
• data-driven discovery