CIDER-FM: Foundation Models for Causal Inference from Diverse Experimental Regimes

2026-09-30

AI Summary

This study addresses the non-uniqueness of causal models derived solely from observational data and the frequent absence of target interventional experiments. To overcome these limitations, this work proposes CIDER-FM, a framework that integrates observational and surrogate interventional data to predict target conditional interventional distributions (CIDs). The approach formalizes the concept of surrogate experiments by introducing intervention-aware representations and a hierarchical three-axis attention mechanism, while constructing a causal foundation model grounded in synthetic structural causal model priors. Extensive evaluations across multiple benchmarks demonstrate superior CID prediction performance. The results confirm that incorporating experimental context significantly outperforms purely observational modeling, thereby establishing a novel paradigm for causal inference.

Abstract

Causal foundation models (CFMs) amortise causal inference over priors of synthetic structural causal models (SCMs), predicting the effect of an experiment on a specific variable. However, observational data alone may leave multiple causal models compatible with available evidence, while experimental data with interventions on exactly the variable of interest might be unavailable. This work studies CFMs as a method to combine finite observational and surrogate-interventional datasets in order to predict a target conditional interventional distribution (CID) more accurately than with observational data alone. We first formalise the conceptual benefits of surrogate experiments. Building on this analysis, we introduce \textsc{Foundation Models for Causal Inference from Diverse Experimental Regimes} (\emph{CIDER-FM}), a causal foundation model that uses an intervention-aware representation and hierarchical three-axis attention to exchange information across variables, samples, and experimental regimes. We evaluate CIDER-FM against a wide range of baselines across diverse synthetic graph and mechanism families, as well as on both simulated and real-world data from Causal Chambers. Our results demonstrate strong CID prediction performance and show that incorporating experimental context can improve predictions over observational data alone.

Problem

• Causal Foundation Models
• Causal Inference
• Conditional Interventional Distribution
• Surrogate Interventions
• Observational Data

Innovation

• Causal Foundation Models
• Surrogate Interventions
• Intervention-aware Representation
• Hierarchical Three-axis Attention
• Conditional Interventional Distribution