Conformal Inference for Counterfactuals and Individual Treatment Effects with Experiment Attrition

2026-04-01

AI Summary

This study addresses the challenge of causal inference bias arising from sample attrition in surveys and field experiments by introducing conformal inference into settings with missing data. It proposes a unified framework that integrates counterfactual modeling, weighting, and imputation strategies to estimate individual treatment effects without relying on strong, untestable assumptions commonly required by traditional methods. The approach yields prediction intervals that are both robust and precise, enabling valid comparisons of treatment effects across retained participants, those lost to follow-up, and the full sample. Simulation studies demonstrate that the proposed framework achieves higher coverage rates while producing narrower intervals compared to existing methods. Reanalyses of two empirical datasets further reveal heterogeneous treatment effects across distinct subpopulations, underscoring the method’s practical utility.

Technology Category

• Machine Learning: Causal Learning
• Reasoning under Uncertainty: Causality
• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning

Application Category

• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

Attrition in survey and field experiments presents a challenge for social science research. Common approaches to deal with this problem -- such as complete case analysis, multiple imputation, and weighting methods -- rely on strong assumptions that may not hold in practice. This paper introduces a new method that combines recent advances in statistical inference with established tools for handling missing data. The approach produces prediction intervals for treatment effects that are both robust and precise. Evidence from simulation studies shows that the method achieves better coverage and produces narrower intervals than common alternatives. The reanalysis of two recently published experiment studies illustrates how this framework allows researchers to compare treatment effects across participants who remain in the study, those who drop out, and the full sample. Taken together, these results highlight how the proposed approach provides a stronger foundation for causal inference in the presence of attrition.

Problem

• attrition
• counterfactuals
• individual treatment effects
• causal inference
• missing data

Innovation

• conformal inference
• counterfactuals
• individual treatment effects
• experiment attrition
• missing data