Tailoring Strictly Proper Scoring Rules for Downstream Tasks: An Application to Causal Inference

2026-06-02

AI Summary

Traditional probabilistic models are typically trained using task-agnostic log-loss, which often yields propensity scores with large errors, high bias, and high variance in boundary regions—particularly detrimental in causal inference tasks such as inverse probability weighting. This work proposes a general framework that, for the first time, integrates the error structure of downstream tasks into the design of strictly proper scoring rules. By aligning the local curvature of the scoring rule with that of the target loss, the authors derive a closed-form loss function tailored for average treatment effect estimation, along with its associated canonical probability mapping, enabling end-to-end task-oriented training. The approach is compatible with both neural networks and gradient boosting models and consistently outperforms standard log-likelihood and covariate balancing methods across multiple causal inference benchmarks, substantially improving estimation accuracy and stability.

Abstract

Probabilistic models are typically trained using task-agnostic objectives like log-loss, which can lead to significant errors in downstream estimation. This disconnect is especially critical in Inverse Probability Weighting (IPW) for causal inference, where propensity score errors near $0$ and $1$ often lead to high bias and variance. We propose a principled framework for deriving task-specific strictly proper scoring rules by matching the local curvature of the downstream error metric. We apply this to the Average Treatment Effect (ATE) estimation, deriving a closed-form loss and its corresponding canonical probability mapping that can be readily integrated with any model like a neural network or a gradient boosting algorithm. Extensive evaluations on causal inference benchmarks demonstrate that our tailored objective consistently outperforms standard likelihood-based and covariate-balancing approaches.

Problem

• causal inference
• inverse probability weighting
• propensity score
• average treatment effect
• strictly proper scoring rules

Innovation

• strictly proper scoring rules
• task-specific loss
• inverse probability weighting
• causal inference
• average treatment effect