Beyond Demographic Balance: Multi-Metric and Intersectional Evaluation of Fairness in MIMIC-IV Mortality Prediction

2026-10-01

AI Summary

This study addresses the vulnerability of fairness evaluation in clinical prediction models to metric selection and demographic granularity, where existing methods struggle to capture intersectional subgroup heterogeneity. Focusing on ICU mortality prediction using MIMIC-IV, this work proposes a lightweight, outcome-independent joint balancing strategy to synergistically improve representation across race, sex, and insurance dimensions. Furthermore, it constructs a multi-metric framework encompassing accuracy, AUROC, sensitivity, and false positive rate to enable error-rate-independent, fine-grained evaluation at both marginal and three-way intersectional subgroup levels. The findings reveal substantial discrepancies in intervention effect assessments under single metrics and demonstrate that marginal aggregation can obscure heterogeneous error distributions within intersectional subgroups, thereby underscoring the necessity of multidimensional fairness evaluation.

Abstract

Fairness conclusions in clinical prediction can depend strongly on both the metrics reported and the demographic resolution at which performance is evaluated. We revisit these evaluation choices for ICU mortality prediction on MIMIC-IV, comparing predictive-utility and subgroup-error metrics across several fairness interventions. As a complementary case study, we introduce a lightweight adaptation strategy that jointly balances ethnicity--gender--insurance representation without conditioning on mortality outcomes, allowing demographic representation balancing to be examined separately from outcome-conditioned or direct error-rate interventions. We evaluate its behavior at both marginal and corresponding three-way intersectional subgroup levels, while accounting for the statistical support of finer-grained estimates. The results show that interventions can receive substantially different assessments across accuracy/AUROC, sensitivity, and false-positive rate, and that marginal demographic summaries can conceal heterogeneous error profiles within their constituent intersections, including among larger subgroups. These findings highlight the importance of evaluating fairness interventions at both complementary metric and subgroup resolutions, while accounting for the intervention target and the reliability of subgroup estimates.

Problem

• fairness evaluation
• intersectionality
• mortality prediction
• multi-metric assessment
• health equity

Innovation

• Intersectional Fairness
• Lightweight Adaptation Strategy
• Multi-Metric Evaluation
• MIMIC-IV
• Demographic Representation Balancing