From Training to Deployment: Post-Hoc Causal Feature Identification via Sensitivity Ratios

2026-07-28

AI Summary

This study addresses the challenge of distinguishing causal features from spurious ones in pre-trained models without access to training dynamics. The authors propose Normalized Sensitivity Ratio (NSR), a post-hoc, model-agnostic diagnostic that identifies causal features by evaluating the stability of feature sensitivities across environments under structured distribution shifts. Grounded in linear structural causal models, NSR constructs a theoretically justified metric based on the coefficient of variation of sensitivities, enabling precise identification of causal features without any training information and providing formal characterization of its failure conditions. Empirical results demonstrate near-perfect performance on synthetic data satisfying model assumptions (AUROC = 1.000), strong consistency in feature rankings across five diverse model architectures (Kendall’s τ ≥ 0.529), and high precision in real-world settings (Precision@7 = 0.75 on bike-sharing data).

Abstract

Given a model that is already trained, which features does it rely on causally versus spuriously? Existing methods require access to the training procedure and cannot answer this post-hoc. We introduce the \textbf{Normalised Sensitivity Ratio~(NSR)}, a post-hoc, model-agnostic diagnostic for this question under a structured-shift regime: environments differ primarily in the mean of spurious features while the causal mechanism and causal marginals remain stable, as in multi-site clinical data or multi-batch genomics. Within this regime, causal features induce constant model sensitivity across environments while spurious features track shift. NSR formalises this as the squared coefficient of variation of per-environment sensitivity. Under a linear structural causal model (SCM) with $K\ge3$ non-degenerate environments, NSR achieves exact identification (Theorem~1). We fully characterise failure: weak shifts ($O(\varepsilon^4)$ collapse), degenerate geometry, and proxy attenuation ($O((1-α)^4)$), giving practitioners quantitative criteria for assessing whether the regime holds. Finite-sample rates are $O_p(n^{-1})$ under the null and $O_p(n^{-1/2})$ under the alternative. Experiments confirm all theoretical predictions on synthetic data (area under the ROC curve [AUROC] $= 1.000$ under conditions satisfying the regime), show consistent rankings across five model families (Kendall $τ\ge0.529$), and recover six of eight causal features on bike-sharing data (Precision@7 $= 0.75$) without modifying any trained model.

Problem

• causal feature identification
• post-hoc analysis
• spurious correlation
• model interpretability
• distribution shift

Innovation

• Post-hoc Causal Identification
• Normalised Sensitivity Ratio
• Model-agnostic Diagnostics
• Structured Distribution Shift
• Spurious Feature Detection