On the relationship between Koopman operator approximations and neural ordinary differential equations for data-driven time-evolution predictions

2024-11-20

arXiv.org

AI Summary

This work investigates the intrinsic unification between Koopman operator approximation and neural ordinary differential equations (Neural ODEs) in data-driven modeling of nonlinear dynamical systems. We propose an extended dynamic mode decomposition framework with deep learning–enhanced nonlinear state-space projection (EDMD-DL), and rigorously prove for the first time that its continuous-time limit is mathematically equivalent to a Neural ODE structure. By jointly optimizing dictionary learning and differentiable projection, our approach achieves a unified paradigm for both discrete- and continuous-time modeling. Evaluations on chaotic systems—including the Lorenz system and a nine-mode turbulent flow—demonstrate that EDMD-DL matches state-of-the-art non-Markovian models in short-term trajectory prediction, long-term statistical reconstruction, and rare-event capture, while exhibiting high accuracy and robustness. The method establishes a new theoretical foundation and practical pathway for interpretable neural dynamical modeling.

Technology Category

Machine Learning: Learning with ManifoldsNatural Language Processing: Learning & Optimization for NLPSearch and Optimization: Mixed Discrete/Continuous Search

Application Category

Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphsEconomics, Online Markets and Human Computation: Cost models of using LLMs in production systemsSemantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

This work explores the relationship between state space methods and Koopman operator-based methods for predicting the time-evolution of nonlinear dynamical systems. We demonstrate that extended dynamic mode decomposition with dictionary learning (EDMD-DL), when combined with a state space projection, is equivalent to a neural network representation of the nonlinear discrete-time flow map on the state space. We highlight how this projection step introduces nonlinearity into the evolution equations, enabling significantly improved EDMD-DL predictions. With this projection, EDMD-DL leads to a nonlinear dynamical system on the state space, which can be represented in either discrete or continuous time. This system has a natural structure for neural networks, where the state is first expanded into a high dimensional feature space followed by a linear mapping which represents the discrete-time map or the vector field as a linear combination of these features. Inspired by these observations, we implement several variations of neural ordinary differential equations (ODEs) and EDMD-DL, developed by combining different aspects of their respective model structures and training procedures. We evaluate these methods using numerical experiments on chaotic dynamics in the Lorenz system and a nine-mode model of turbulent shear flow, showing comparable performance across methods in terms of short-time trajectory prediction, reconstruction of long-time statistics, and prediction of rare events. These results highlight the equivalence of the EDMD-DL implementation with a state space projection to a neural ODE representation of the dynamics. We also show that these methods provide comparable performance to a non-Markovian approach in terms of prediction of extreme events.

Problem

• Explores relationship between Koopman operator and neural ODEs.
• Improves EDMD-DL predictions via state space projection.
• Compares methods for chaotic dynamics and rare event prediction.

Innovation

• Combines EDMD-DL with state space projection
• Introduces nonlinearity via neural network mapping
• Evaluates neural ODEs and EDMD-DL on chaotic systems