🤖 AI Summary
Modeling temporal structure in infinite-dimensional dynamical systems—such as solution maps of stochastic differential equations (SDEs)—remains challenging due to the lack of expressive, causally consistent architectures operating on Banach spaces.
Method: This paper introduces the *causal neural operator* framework, the first to achieve uniform approximation of Hölder-continuous or smooth trace-class causal operators on Banach spaces. It integrates a sequence architecture grounded in infinite-dimensional linear metric spaces, explicit causal constraints, functional approximation theory, and stochastic operator theory.
Contributions/Results: We establish a quantitative trade-off between latent state dimension and approximation accuracy, substantially improving error bounds for RNNs in dynamical system approximation. We prove uniform approximability over arbitrary finite time horizons and compact sets, with error rates superior to classical feedforward-network-based results. By explicitly encoding temporal geometry and causality, our framework overcomes a key limitation of conventional neural operators—which neglect sequential structure—and establishes a new paradigm for causality-driven modeling of infinite-dimensional dynamics.
📝 Abstract
Several non-linear operators in stochastic analysis, such as solution maps to stochastic differential equations, depend on a temporal structure which is not leveraged by contemporary neural operators designed to approximate general maps between Banach space. This paper therefore proposes an operator learning solution to this open problem by introducing a deep learning model-design framework that takes suitable infinite-dimensional linear metric spaces, e.g. Banach spaces, as inputs and returns a universal extit{sequential} deep learning model adapted to these linear geometries specialized for the approximation of operators encoding a temporal structure. We call these models extit{Causal Neural Operators}. Our main result states that the models produced by our framework can uniformly approximate on compact sets and across arbitrarily finite-time horizons H""older or smooth trace class operators, which causally map sequences between given linear metric spaces. Our analysis uncovers new quantitative relationships on the latent state-space dimension of Causal Neural Operators, which even have new implications for (classical) finite-dimensional Recurrent Neural Networks. In addition, our guarantees for recurrent neural networks are tighter than the available results inherited from feedforward neural networks when approximating dynamical systems between finite-dimensional spaces.
