🤖 AI Summary
Traditional RNNs and Transformers struggle to model continuous-time dynamics and irregularly sampled time series. Method: This paper systematically investigates neural differential equations (NDEs)—including neural ordinary differential equations (ODEs), controlled differential equations (CDEs), and stochastic differential equations (SDEs)—for time-series analysis, unifying their mathematical frameworks, adaptive numerical solvers (e.g., Dopri5), adjoint sensitivity methods, and controlled path theory. Contribution/Results: We propose a comprehensive taxonomy covering differentiable simulation, missing-value imputation, and extrapolative forecasting, identifying optimal technical pathways for each task. We establish the paradigmatic advantage of NDEs over discrete models in capturing continuous dynamics, while highlighting scalability and numerical stability as two fundamental challenges. This work provides both theoretical foundations and practical design principles for deploying NDEs in real-world time-series applications.
📝 Abstract
Time series modeling and analysis has become critical in various domains. Conventional methods such as RNNs and Transformers, while effective for discrete-time and regularly sampled data, face significant challenges in capturing the continuous dynamics and irregular sampling patterns inherent in real-world scenarios. Neural Differential Equations (NDEs) represent a paradigm shift by combining the flexibility of neural networks with the mathematical rigor of differential equations. This paper presents a comprehensive review of NDE-based methods for time series analysis, including neural ordinary differential equations, neural controlled differential equations, and neural stochastic differential equations. We provide a detailed discussion of their mathematical formulations, numerical methods, and applications, highlighting their ability to model continuous-time dynamics. Furthermore, we address key challenges and future research directions. This survey serves as a foundation for researchers and practitioners seeking to leverage NDEs for advanced time series analysis.