🤖 AI Summary
This work addresses the complexity of modeling high-dimensional continuous-time series by proposing a unified framework based on Neural Stochastic Differential Equations (Neural SDEs). Methodologically, it treats observed sequences as discrete samples from an underlying continuous dynamical system, jointly parameterizing both drift and diffusion terms, and introduces a numerically simulation-free maximum-likelihood training paradigm that integrates stochastic calculus with deep neural networks. Key contributions include: (i) the first systematic empirical validation of the superiority of continuous-time SDE modeling over discrete-time alternatives in high-dimensional sequential tasks—particularly in embodied intelligence and generative AI; and (ii) a novel, efficient, differentiable training algorithm that avoids pathwise simulation. Experiments demonstrate state-of-the-art performance across multiple continuous-domain sequence modeling benchmarks, with significant improvements in long-horizon prediction stability and generative sample diversity.
📝 Abstract
Inspired by the ubiquitous use of differential equations to model continuous dynamics across diverse scientific and engineering domains, we propose a novel and intuitive approach to continuous sequence modeling. Our method interprets time-series data as extit{discrete samples from an underlying continuous dynamical system}, and models its time evolution using Neural Stochastic Differential Equation (Neural SDE), where both the flow (drift) and diffusion terms are parameterized by neural networks. We derive a principled maximum likelihood objective and a extit{simulation-free} scheme for efficient training of our Neural SDE model. We demonstrate the versatility of our approach through experiments on sequence modeling tasks across both embodied and generative AI. Notably, to the best of our knowledge, this is the first work to show that SDE-based continuous-time modeling also excels in such complex scenarios, and we hope that our work opens up new avenues for research of SDE models in high-dimensional and temporally intricate domains.