Calibration Strategies for Robust Causal Estimation: Theoretical and Empirical Insights on Propensity Score Based Estimators

2025-03-21

AI Summary

This paper addresses the robustness of propensity score estimation in causal inference under challenging settings—including poor covariate overlap, small sample sizes, and class imbalance—and systematically investigates how calibration strategies affect inverse probability weighting (IPW) and double-robust debiased machine learning (DML). Theoretically, it establishes for the first time how calibration improves the bias–variance trade-off in DML, elucidates the role of sample splitting in ensuring valid post-calibration statistical inference, and rigorously proves that calibration simultaneously reduces both bias and variance without compromising double robustness. Empirically, calibration substantially decreases both bias and variance in IPW, enhances the stability and estimation accuracy of DML in small-sample regimes, and seamlessly integrates with flexible learners such as gradient boosting. The work thus bridges theoretical rigor with practical applicability in modern causal machine learning.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Causality
• Intelligent Robots: State Estimation

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Web Mining and Content Analysis: Machine learning and data science for the Web

Abstract

The partitioning of data for estimation and calibration critically impacts the performance of propensity score based estimators like inverse probability weighting (IPW) and double/debiased machine learning (DML) frameworks. We extend recent advances in calibration techniques for propensity score estimation, improving the robustness of propensity scores in challenging settings such as limited overlap, small sample sizes, or unbalanced data. Our contributions are twofold: First, we provide a theoretical analysis of the properties of calibrated estimators in the context of DML. To this end, we refine existing calibration frameworks for propensity score models, with a particular emphasis on the role of sample-splitting schemes in ensuring valid causal inference. Second, through extensive simulations, we show that calibration reduces variance of inverse-based propensity score estimators while also mitigating bias in IPW, even in small-sample regimes. Notably, calibration improves stability for flexible learners (e.g., gradient boosting) while preserving the doubly robust properties of DML. A key insight is that, even when methods perform well without calibration, incorporating a calibration step does not degrade performance, provided that an appropriate sample-splitting approach is chosen.

Problem

• Improving robustness of propensity score estimators in challenging data settings
• Theoretical analysis of calibrated estimators in double machine learning
• Reducing variance and bias in propensity score estimation via calibration

Innovation

• Extends calibration techniques for propensity scores
• Refines calibration frameworks with sample-splitting schemes
• Reduces variance and bias in small-sample regimes