🤖 AI Summary
This study addresses the challenge of detecting information leakage solely from a model’s predictive outputs, without access to training code, external data, or domain knowledge. Framed within decision theory, the approach models leakage diagnosis as a functional of predictive risk and outcome distribution, linking proper scoring rules with decision curve analysis via threshold-weighted associations to enable detection without prior assumptions. The work introduces a novel tripartite classification of information leakage—miscalibration, generalized calibration, and determinism—and theoretically establishes that generalized calibration leakage is fundamentally unidentifiable, whereas near-deterministic subgroups can be efficiently detected. Empirical validation on UK Biobank demonstrates detection of temporal-window comorbidity leakage down to Δc*≈0.007 in under one second, while also revealing inherent structural limitations of purely output-driven leakage detection.
📝 Abstract
Data leakage -- contamination of a model with information unavailable at baseline -- is the dominant reproducibility failure in machine-learning-based science, yet detection tools require training code, external data, or domain expertise. None operates on the artifact an auditor most often holds: the model's output. We ask what can be decided about leakage from predictions and outcomes alone. We give a decision-theoretic framework in which leakage diagnostics are functionals of the predicted-risk/outcome law, parameterized by a threshold-weighting linked to proper scoring rules and decision-curve analysis. We prove a sharp impossibility: a recalibrated leak matching an honest model's calibration and discrimination is indistinguishable from honest performance by \emph{any} function of the predictions, so the broad class is detectable only against an externally supplied ceiling on achievable discrimination. We then prove what leakage cannot hide: a near-deterministic subgroup -- the signature of a near-label leak -- produces a sustained unit-purity head that no legitimate predictor of a non-deterministic outcome can manufacture, yielding a prior-free test. These results organize leakage into a trichotomy -- miscalibrated, broad-calibrated, and deterministic -- each with a matched detector and failure mode. We validate on UK Biobank using time-windowed comorbidity leakage with known, graded severity, measuring a detection floor of $Δ\cstar \approx 0.007$ on this endpoint, below which residual leakage is undetectable from output and too small to alter conclusions. The numerical floor is cohort- and endpoint-specific; the structural lesson is general: output-only detection fails where residual leakage is indistinguishable from an honestly stronger predictor. The test returns a verdict on a prediction vector in under a second on commodity hardware.