Hybrid Meta-learners for Estimating Heterogeneous Treatment Effects

2025-06-16

AI Summary

Existing direct and indirect meta-learners for Conditional Average Treatment Effect (CATE) estimation from observational data suffer from instability and suboptimal bias–variance trade-offs. Method: We propose the H-learner—a unified framework that directly approximates CATE by modeling the difference between intermediate functions, thereby bypassing explicit modeling of potential outcomes. It introduces an adaptive double regularization mechanism that dynamically interpolates between direct and indirect learning paradigms to achieve Pareto-optimal bias–variance balance, and uniquely integrates tree-based models with neural networks, selecting regularization strength in a data-driven manner. Contribution/Results: Evaluated across multiple semi-synthetic and real-world datasets, H-learner consistently achieves state-of-the-art performance, residing on the Pareto frontier of CATE estimation error—significantly outperforming T-, X-, and R-learners.

Technology Category

• Machine Learning: Causal Learning
• Search and Optimization: Learning to Search
• Game Theory and Economic Paradigms: Adversarial Learning

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Economics, Online Markets and Human Computation: Social networks and social learning

Abstract

Estimating conditional average treatment effects (CATE) from observational data involves modeling decisions that differ from supervised learning, particularly concerning how to regularize model complexity. Previous approaches can be grouped into two primary"meta-learner"paradigms that impose distinct inductive biases. Indirect meta-learners first fit and regularize separate potential outcome (PO) models and then estimate CATE by taking their difference, whereas direct meta-learners construct and directly regularize estimators for the CATE function itself. Neither approach consistently outperforms the other across all scenarios: indirect learners perform well when the PO functions are simple, while direct learners outperform when the CATE is simpler than individual PO functions. In this paper, we introduce the Hybrid Learner (H-learner), a novel regularization strategy that interpolates between the direct and indirect regularizations depending on the dataset at hand. The H-learner achieves this by learning intermediate functions whose difference closely approximates the CATE without necessarily requiring accurate individual approximations of the POs themselves. We demonstrate empirically that intentionally allowing suboptimal fits to the POs improves the bias-variance tradeoff in estimating CATE. Experiments conducted on semi-synthetic and real-world benchmark datasets illustrate that the H-learner consistently operates at the Pareto frontier, effectively combining the strengths of both direct and indirect meta-learners.

Problem

• Estimating heterogeneous treatment effects from observational data
• Balancing regularization between direct and indirect meta-learners
• Improving bias-variance tradeoff in CATE estimation

Innovation

• Hybrid Learner interpolates direct and indirect regularizations
• Learns intermediate functions approximating CATE directly
• Improves bias-variance tradeoff by allowing suboptimal PO fits