CalTwin: Towards Calibrated, Shift-Robust Medical World Models via Fisher-Information Regularisation

2026-07-29

AI Summary

This work addresses covariate shift and confidence miscalibration in clinical deployment of medical world models by proposing CalTwin, a lightweight regularization method that uniquely integrates Fisher Information Matrix (FIM) penalization with confidence miscalibration penalization into a GRU-based latent state transition predictor. Through joint optimization, CalTwin simultaneously mitigates performance degradation under out-of-distribution data and overconfidence. Evaluated on the PhysioNet 2019 Sepsis Challenge dataset, CalTwin reduces the mean squared error of out-of-distribution next-step latent state prediction by 9.1%—with 7.0% attributable to FIM regularization—and modestly improves calibration, lowering expected calibration error (ECE) by 0.7%, thereby significantly enhancing model reliability and calibration in multi-center, heterogeneous clinical settings.

Abstract

Medical world models aim to learn a latent state of patient or organ physiology and a transition function that forecasts how that state evolves under interventions, supporting downstream tasks from imaging-based diagnosis to digital-twin treatment planning. Two failure modes threaten the reliability of such models in clinical deployment: (i)~\emph{covariate shift}, because training data are fragmented across hospitals, scanners, and time, so the feature distribution seen by the latent-dynamics predictor differs across fragments and from the distribution at deployment; and (ii)~\emph{confidence misalignment}, because multi-step forecasts are often overconfident exactly where clinical risk is highest. We argue that both problems admit a unified treatment via a single lightweight regularisation objective, \textbf{CalTwin}, which combines a Fisher-Information-based shift penalty adapted from our prior work on fragmented covariate-shift remediation~\cite{khan2025mitigating,khan2025causal} with a Confidence Misalignment Penalty adapted from our prior work on calibrated vision-language classification~\cite{khan2025confidence}, applied here to a GRU-based medical world model's latent transition predictor. We derive the combined objective, establish which proof steps transfer from the classification setting without modification and which require adaptation, and evaluate it on the PhysioNet 2019 Sepsis Challenge, treating the two hospital systems as sequential training fragments and the unseen system as an out-of-distribution test. CalTwin reduces OOD next-step latent-state MSE by 9.1\% relative to the no-penalty baseline (FIM penalty alone accounts for 7.0\%); the ECE reduction from the Confidence Misalignment Penalty is real but small (0.7\% for CalTwin, 1.3\% for CMP alone).

Problem

• covariate shift
• confidence misalignment
• medical world models
• distribution shift
• calibration

Innovation

• Fisher-Information Regularisation
• Covariate Shift
• Confidence Calibration
• Medical World Models
• Digital Twin