A physics-informed transformer neural operator for learning generalized solutions of initial boundary value problems

2024-12-12

arXiv.org

AI Summary

Existing neural operator methods for nonlinear partial differential equation (PDE) initial-boundary value problems rely heavily on large-scale simulation data and fail to generalize to unseen initial or boundary conditions. Method: We propose PINTO—a novel physics-informed neural operator that introduces a cross-attention-based iterative kernel integral operator unit, mapping solution-domain points to initial/boundary-condition-aware representations; it employs a purely physics-informed loss, enabling end-to-end operator learning without any simulation data. Contribution/Results: PINTO achieves zero-shot generalization across initial and boundary conditions. Evaluated on five engineering-relevant PDEs—including Navier–Stokes equations—it reduces relative error on unseen conditions to only 1/5–1/3 of state-of-the-art methods. Moreover, it successfully extrapolates to time steps outside the training horizon, eliminating the need for retraining and breaking the dependency on labeled simulation data.

Technology Category

• Machine Learning: Neuro-Symbolic Learning
• Computer Vision: Diffusion Models for Vision
• Search and Optimization: Learning to Search

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Initial boundary value problems arise commonly in applications with engineering and natural systems governed by nonlinear partial differential equations (PDEs). Operator learning is an emerging field for solving these equations by using a neural network to learn a map between infinite dimensional input and output function spaces. These neural operators are trained using a combination of data (observations or simulations) and PDE-residuals (physics-loss). A major drawback of existing neural approaches is the requirement to retrain with new initial/boundary conditions, and the necessity for a large amount of simulation data for training. We develop a physics-informed transformer neural operator (named PINTO) that efficiently generalizes to unseen initial and boundary conditions, trained in a simulation-free setting using only physics loss. The main innovation lies in our new iterative kernel integral operator units, implemented using cross-attention, to transform the PDE solution's domain points into an initial/boundary condition-aware representation vector, enabling efficient learning of the solution function for new scenarios. The PINTO architecture is applied to simulate the solutions of important equations used in engineering applications: advection, Burgers, and steady and unsteady Navier-Stokes equations (three flow scenarios). For these five test cases, we show that the relative errors during testing under challenging conditions of unseen initial/boundary conditions are only one-fifth to one-third of other leading physics informed operator learning methods. Moreover, our PINTO model is able to accurately solve the advection and Burgers equations at time steps that are not included in the training collocation points. The code is available at https://github.com/quest-lab-iisc/PINTO

Problem

• Learning solutions for initial boundary value problems in nonlinear PDEs
• Generalizing to unseen initial and boundary conditions without retraining
• Reducing reliance on simulation data by using physics loss

Innovation

• Physics-informed transformer neural operator (PINTO)
• Iterative kernel integral operator units
• Simulation-free training with physics loss