Sensitivity Analysis of the Consistency Assumption

2025-12-24

AI Summary

In causal inference, the consistency assumption—that potential outcomes under treatment coincide with observed outcomes, and that there are no hidden versions of treatment—is often violated, particularly due to unobserved treatment heterogeneity (e.g., variation in surgeons’ skill levels), rather than classical unmeasured confounding. Method: We formally distinguish treatment versions from covariates and develop the first sensitivity analysis framework specifically tailored to hidden treatment versions. Leveraging a novel notation system, we integrate causal diagrams with counterfactual models to define parameterized sensitivity measures and derive bounds on causal effects. Contribution/Results: Our framework is empirically validated in real-world surgical settings. It quantifies how violations of consistency bias causal effect estimates and substantially enhances the rigor of robustness assessment for causal conclusions—enabling principled evaluation of estimation uncertainty arising from treatment-version heterogeneity.

Technology Category

• Reasoning under Uncertainty: Causality
• Machine Learning: Causal Learning
• Knowledge Representation and Reasoning: Diagnosis and Abductive Reasoning

Application Category

• Responsible Web: Human-perceived consequences of algorithmic deployment on the web
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Social Networks and Social Media: Fairness and bias in social network and social media analysis

Abstract

Sensitivity analysis informs causal inference by assessing the sensitivity of conclusions to departures from assumptions. The consistency assumption states that there are no hidden versions of treatment and that the outcome arising naturally equals the outcome arising from intervention. When reasoning about the possibility of consistency violations, it can be helpful to distinguish between covariates and versions of treatment. In the context of surgery, for example, genomic variables are covariates and the skill of a particular surgeon is a version of treatment. There may be hidden versions of treatment, and this paper addresses that concern with a new kind of sensitivity analysis. Whereas many methods for sensitivity analysis are focused on confounding by unmeasured covariates, the methodology of this paper is focused on confounding by hidden versions of treatment. In this paper, new mathematical notation is introduced to support the novel method, and example applications are described.

Problem

• Assesses sensitivity to hidden treatment versions
• Introduces new notation for novel sensitivity analysis
• Focuses on confounding by hidden treatment versions

Innovation

• Novel sensitivity analysis for hidden treatment versions
• Introduces new mathematical notation for methodology
• Focuses on confounding by hidden treatment versions