Towards Continuous-time Causal Foundation Models

2026-05-26

AI Summary

Existing continuous-time causal models still rely on discrete observations, causing trajectory distributions to depend on sampling times and hindering truly continuous modeling. This work proposes the first continuous-time causal foundation model whose trajectory distribution is invariant to observation scheduling. The approach introduces a three-tier taxonomy, with the top tier combining random DAGs with either Ornstein-Uhlenbeck processes or small MLP-based nonlinear drift terms, alongside fine-grained numerical integration and a decoupled observation mechanism. By relaxing the discrete-time Markov assumption, the model accommodates irregular sampling and diverse interventions. Experiments across all eight evaluation settings demonstrate that fine-grid integration significantly outperforms baselines (sign test p < 1/256), with performance improving as the evaluation grid is refined. The authors also release corresponding prior models and a zero-shot evaluation protocol.

Abstract

Extending discrete-time causal Prior-data Fitted Networks for time series to continuous time invites writing the mechanism as a stochastic differential equation (SDE) -- but if the SDE is integrated \emph{once per observation gap}, the trajectory law depends on when it is observed, and the prior remains a discrete-time Markov model in SDE clothing. We propose a precise continuity criterion -- trajectory-law invariance to the observation schedule -- together with a three-tier taxonomy (discrete; naive observation-grid integration; fine-grid integration with decoupled observation) and a construction realising the top tier on a random DAG with OU or small-MLP nonlinear drifts, irregular observation schedules, and hard / soft / time-varying interventions. A $2 \times 2$ encoder $\times$ integrator ablation, run independently on a linear and a nonlinear prior, finds fine-grid integration beats naive on 8/8 cells (sign-consistency $p < 1/256$) with the gap growing as the eval grid refines; the encoder axis is null with fine integration but time-aware-leading with naive. We release the prior and a preliminary zero-shot protocol on pharmacokinetic and physical-system data.

Problem

• continuous-time causal models
• trajectory-law invariance
• stochastic differential equations
• irregular observation schedules
• causal foundation models

Innovation

• continuous-time causal models
• trajectory-law invariance
• stochastic differential equations
• fine-grid integration
• causal foundation models