Lyapunov weights to convey the meaning of time in physics-informed neural networks

2024-07-31

arXiv.org

AI Summary

To address training bias in Physics-Informed Neural Networks (PINNs) arising from the intrinsic temporal characteristics when solving time-dependent differential equations, this work proposes a Lyapunov exponent (LE)-based adaptive temporal weighting mechanism. We theoretically establish, for the first time, the causal relationship between LEs and PINN temporal weights, and derive an optimal cumulative LE-integral weighting formulation under computational constraints—ensuring both physical interpretability and dynamical adaptivity. The method is hyperparameter-free and universally applicable to chaotic, periodic, and asymptotically stable systems. Extensive experiments across diverse dynamical systems demonstrate substantial improvements in prediction accuracy and convergence stability; notably, prediction error is reduced by up to 47% in strongly chaotic regimes.

Technology Category

• Planning, Routing, and Scheduling: Temporal Planning
• Machine Learning: Causal Learning
• Cognitive Modeling & Cognitive Systems: Adaptive Behavior

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Time is not a dimension as the others. In Physics-Informed Neural Networks (PINN) several proposals attempted to adapt the time sampling or time weighting to take into account the specifics of this special dimension. But these proposals are not principled and need guidance to be used. We explain here theoretically why the Lyapunov exponents give actionable insights and propose a weighting scheme to automatically adapt to chaotic, periodic or stable dynamics. We characterize theoretically the best weighting scheme under computational constraints as a cumulative exponential integral of the local Lyapunov exponent estimators and show that it performs well in practice under the regimes mentioned above.

Problem

• Improving time sampling for PINNs in differential equations
• Automatically adjusting weights based on stability regimes
• Enhancing convergence and accuracy without hyperparameter tuning

Innovation

• Time weighting based on Lyapunov exponents
• Automatic weight adjustment by stability regime
• Improved convergence without hyperparameter tuning