The Fourier Spectral Transformer Networks For Efficient and Generalizable Nonlinear PDEs Prediction

2025-07-07

AI Summary

To address the low accuracy and poor generalization of long-term predictions for nonlinear partial differential equations (PDEs) under limited training data, this paper proposes a spectral-domain Transformer framework. First, the PDE is transformed via Fourier spectral decomposition into a system of ordinary differential equations (ODEs) governing the temporal evolution of spectral coefficients. High-order ODE solvers are then employed to generate high-fidelity synthetic training data. Finally, a lightweight Transformer architecture is designed to model the spatiotemporal dynamics of these spectral coefficients. The method innovatively integrates the physical fidelity of classical spectral methods with the long-range dependency modeling capability of self-attention mechanisms. Experiments on the two-dimensional incompressible Navier–Stokes equations and the one-dimensional Burgers equation demonstrate superior long-term prediction accuracy and generalization over traditional numerical schemes and state-of-the-art data-driven models—particularly under severe data scarcity.

Technology Category

Computer Vision: Diffusion Models for VisionMachine Learning: Deep Generative Models & AutoencodersNatural Language Processing: Code Generation / Program Synthesis from Natural Language

Application Category

Search and Retrieval-Augmented AI: Vertical and domain-specific searchWeb Mining and Content Analysis: Large pretrained models with web dataGraph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs

Abstract

In this work we propose a unified Fourier Spectral Transformer network that integrates the strengths of classical spectral methods and attention based neural architectures. By transforming the original PDEs into spectral ordinary differential equations, we use high precision numerical solvers to generate training data and use a Transformer network to model the evolution of the spectral coefficients. We demonstrate the effectiveness of our approach on the two dimensional incompressible Navier-Stokes equations and the one dimensional Burgers' equation. The results show that our spectral Transformer can achieve highly accurate long term predictions even with limited training data, better than traditional numerical methods and machine learning methods in forecasting future flow dynamics. The proposed framework generalizes well to unseen data, bringing a promising paradigm for real time prediction and control of complex dynamical systems.

Problem

• Efficient prediction of nonlinear PDEs using spectral methods and Transformers
• Generalizable framework for real-time complex dynamical systems control
• Accurate long-term forecasting of flow dynamics with limited data

Innovation

• Integrates spectral methods and Transformer networks
• Transforms PDEs into spectral ODEs for training
• Achieves accurate predictions with limited data