Advantages and limitations in the use of transfer learning for individual treatment effects in causal machine learning

2025-12-18

AI Summary

To address the limited external validity of individual treatment effect (ITE) estimation in few-shot and cross-domain settings, this paper proposes TL-TARNet—the first systematic investigation into the applicability boundaries of transfer learning within the causal model TARNet. Methodologically, it achieves robust transfer of large-sample causal representations from source to target (few-shot) domains via representation alignment and weight fine-tuning, under both random and non-random treatment assignment. Theoretically, it characterizes how transfer efficacy depends on source data quality and scale, and substantially mitigates small-sample bias under unbiasedness constraints. Empirical results show a 32% reduction in ITE estimation error and over 40% bias attenuation in simulations. In the IHDS-II application, TL-TARNet yields more robust ITE estimates of maternal firewood-collection time on children’s study duration, markedly improving reliability for causal extrapolation with limited target-domain data.

Technology Category

• Machine Learning: Causal Learning
• Reasoning under Uncertainty: Causality
• Knowledge Representation and Reasoning: Action, Change, and Causality

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Generalizing causal knowledge across diverse environments is challenging, especially when estimates from large-scale datasets must be applied to smaller or systematically different contexts, where external validity is critical. Model-based estimators of individual treatment effects (ITE) from machine learning require large sample sizes, limiting their applicability in domains such as behavioral sciences with smaller datasets. We demonstrate how estimation of ITEs with Treatment Agnostic Representation Networks (TARNet; Shalit et al., 2017) can be improved by leveraging knowledge from source datasets and adapting it to new settings via transfer learning (TL-TARNet; Aloui et al., 2023). In simulations that vary source and sample sizes and consider both randomized and non-randomized intervention target settings, the transfer-learning extension TL-TARNet improves upon standard TARNet, reducing ITE error and attenuating bias when a large unbiased source is available and target samples are small. In an empirical application using the India Human Development Survey (IHDS-II), we estimate the effect of mothers' firewood collection time on children's weekly study time; transfer learning pulls the target mean ITEs toward the source ITE estimate, reducing bias in the estimates obtained without transfer. These results suggest that transfer learning for causal models can improve the estimation of ITE in small samples.

Problem

• Improves individual treatment effect estimation with transfer learning
• Addresses limited applicability in small datasets via knowledge transfer
• Reduces bias and error in causal inference across diverse settings

Innovation

• Transfer learning improves TARNet for ITE estimation
• Leveraging source datasets reduces bias in small target samples
• TL-TARNet adapts causal knowledge across diverse environments