neural ode modeling

This work addresses the lack of theoretical understanding regarding the online learning dynamics and generalization mechanisms of high-dimensional controlled nonlinear dynamical systems, such as neural ordinary differential equations (neural ODEs). For the first time, it systematically applies dynamical mean-field theory to analyze online stochastic gradient descent training of neural ODEs. In the high-dimensional limit, the framework rigorously solves the coupled dynamics of training and inference and analytically derives the associated learning curves. This study establishes the first tractable theoretical framework for understanding both the training dynamics and generalization capabilities of deep continuous models, revealing the precise evolution laws governing high-dimensional neural ODEs under online learning.

Existing machine learning approaches for dynamical systems are limited by discrete-time modeling and local analysis, failing to capture coexisting local and global bifurcations. To address this, we propose a continuous-time modeling paradigm based on Neural Ordinary Differential Equations (Neural ODEs). Our method directly learns parameter-dependent vector fields from noisy, sparse time-series data, enabling differentiable modeling and extrapolation across bifurcations. This work is the first to apply Neural ODEs to bifurcation structure prediction—overcoming constraints of the training parameter domain—and accurately reconstructs complex bifurcation diagrams in predator–prey systems. It demonstrates robust generalization under data scarcity and noise corruption. The core contribution is the establishment of the first data-driven, continuous-time framework for cross-bifurcation dynamics, uniquely integrating physical interpretability with global predictive capability.

This work addresses the complexity of modeling high-dimensional continuous-time series by proposing a unified framework based on Neural Stochastic Differential Equations (Neural SDEs). Methodologically, it treats observed sequences as discrete samples from an underlying continuous dynamical system, jointly parameterizing both drift and diffusion terms, and introduces a numerically simulation-free maximum-likelihood training paradigm that integrates stochastic calculus with deep neural networks. Key contributions include: (i) the first systematic empirical validation of the superiority of continuous-time SDE modeling over discrete-time alternatives in high-dimensional sequential tasks—particularly in embodied intelligence and generative AI; and (ii) a novel, efficient, differentiable training algorithm that avoids pathwise simulation. Experiments demonstrate state-of-the-art performance across multiple continuous-domain sequence modeling benchmarks, with significant improvements in long-horizon prediction stability and generative sample diversity.

Deep neural networks suffer from weak theoretical foundations, poor interpretability, and limited generalization capacity. To address these challenges, this work introduces a dynamical systems modeling paradigm grounded in differential equations: forward propagation is formulated as a continuous-time dynamic process governed by ordinary differential equations (ODEs) or stochastic differential equations (SDEs). By integrating numerical integration schemes, stability constraints, and path regularization, we design dynamic network architectures that are both theoretically grounded and structurally interpretable. Experiments demonstrate substantial improvements in model stability and out-of-distribution generalization on image classification and time-series forecasting benchmarks. Moreover, the framework enables gradient-based attribution analysis for enhanced interpretability. This study establishes a principled continuous-time design methodology for deep learning, advancing the development of trustworthy intelligent computing systems.

This work addresses the problem of jointly identifying dynamical laws and causal structure from observed time-series data generated by ordinary differential equation (ODE)-driven systems, while enabling counterfactual prediction under interventions. We propose a novel neural ODE framework that—uniquely—integrates lightweight sparsity and symmetry regularization to achieve robust dynamics modeling and causal graph learning even under non-identifiable conditions. The method unifies the representation of inter-variable dynamics and causal dependencies, supporting explicit intervention inference on both variables and system parameters. Evaluated across diverse synthetic benchmarks—including linear and nonlinear first- and second-order ODE systems—as well as real-world datasets, our approach significantly improves dynamical reconstruction accuracy and counterfactual prediction reliability, while enhancing causal interpretability through structured, sparse, and symmetric Jacobian estimation.

This work proposes Dynamic Physical Modeling Neural Networks (DynPMNNs), a novel architecture that enhances model interpretability and representational capacity by integrating principles from physical dynamical systems. Specifically, the hidden layers are formulated as solutions to ordinary differential equations (ODEs), replacing conventional static activation functions with time-evolving dynamical systems—such as the biologically inspired FitzHugh–Nagumo model—and embedded within an end-to-end trainable framework grounded in Reproducing Kernel Banach Space (RKBS) theory. This study presents the first instantiation of such dynamical models within neural network hidden layers and establishes theoretical connections to classical architectures under the RKBS formalism. Empirical evaluation on the California Housing dataset demonstrates that DynPMNNs achieve performance comparable to Neural ODEs and CfCs while using significantly fewer parameters, thereby confirming their modeling efficacy and expressive efficiency.

This study investigates the limited generalization capability of neural ordinary differential equations (nODEs) across graph structures and scales in complex networks. Building upon the Barabási–Barzel vector field, the authors construct an nODE model trained on five canonical classes of graph dynamical systems and systematically evaluate its performance using realistic networks generated via S¹ random geometric graphs with tunable structural properties. The work reveals, for the first time, that degree heterogeneity and the type of dynamical system are the primary factors governing nODE generalization across graphs, while the average clustering coefficient plays a secondary role. The model also demonstrates robustness to missing data and in capturing fixed points. These findings highlight the strong modeling capacity of nODEs on real-world graph structures, yet underscore significant generalization challenges posed by high degree heterogeneity and elevated clustering.

This work addresses the limitations of conventional discrete-time approaches in accurately capturing the instantaneous nonlinear dynamics of electroencephalography (EEG) signals and their tendency to accumulate prediction errors. To overcome these challenges, we propose the first continuous-time modeling framework that integrates Neural Ordinary Differential Equations (Neural ODEs) with the graph structure of EEG data. By embedding spatiotemporal-frequency features into spectral graph nodes, our method enables high-fidelity modeling of the brain’s continuous latent dynamics. The framework supports inference of brain states at arbitrary time points and demonstrates superior performance over existing methods in EEG dynamic prediction tasks, exhibiting enhanced robustness and generalization capability.

In theoretical neuroscience, high-dimensional dynamical models—such as RNNs—face two key challenges: (1) lack of comparability across models due to non-identical dynamical behaviors, and (2) difficulty in identifying critical low-dimensional dynamical motifs (e.g., limit cycles, saddle sets). To address these, we propose DFORM, the first framework leveraging diffeomorphic vector field alignment to rigorously assess topological equivalence of dynamical systems via differentiable, nonlinear coordinate transformations that induce one-to-one trajectory mapping. DFORM integrates neural ODE modeling, trajectory alignment loss, and topology-consistent regularization, enabling unsupervised discovery of low-dimensional invariant manifolds and salient dynamical motifs embedded in high-dimensional systems. Validated on canonical dynamical systems, trained RNNs, and fMRI-informed models, DFORM accurately recovers ground-truth linear and nonlinear coordinate transformations, quantifies topological similarity, and extracts limit cycles consistent with numerical bifurcation analysis.

This work addresses the long-standing lack of a systematic theoretical foundation for deep neural networks (DNNs), which has hindered their interpretability and controllable advancement. By innovatively adopting an ordinary differential equation (ODE) perspective, the study integrates dynamical systems theory and numerical analysis to formulate a continuous-time modeling framework for DNNs. It rigorously establishes correspondence between differential equations and network architectures at both the global structural and individual layer levels. This unified theoretical framework provides, for the first time, a coherent explanation of DNN design principles, performance characteristics, and optimization mechanisms. Consequently, it substantially enhances model interpretability and opens new avenues for algorithmic improvements and cross-domain applications.