Reinforcement Learning Closures for Underresolved Partial Differential Equations using Synthetic Data

2025-05-16

AI Summary

In partial differential equation (PDE) coarse-grid simulations under computational resource constraints, unresolved spatiotemporal interactions degrade accuracy, while conventional closure models suffer from physical inconsistency and strong data dependency. Method: This work proposes a novel closure modeling paradigm integrating manufactured solutions with proximal policy optimization (PPO)—a deep reinforcement learning (DRL) algorithm—marking the first application of RL to PDE closure modeling. High-fidelity synthetic data are generated via manufactured solutions, and physics-informed constraints ensure training stability. The model inherently generalizes across homogeneity classes (non-homogeneous → homogeneous). Results: Evaluated on 1D/2D Burgers equations and the 2D advection equation, the method achieves significant accuracy improvements in coarse-grid simulations using only minimal training data—demonstrating robustness in sparse-observation regimes.

Technology Category

• Search and Optimization: Learning to Search
• Computer Vision: Diffusion Models for Vision
• Machine Learning: Imitation Learning & Inverse Reinforcement Learning

Application Category

• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Partial Differential Equations (PDEs) describe phenomena ranging from turbulence and epidemics to quantum mechanics and financial markets. Despite recent advances in computational science, solving such PDEs for real-world applications remains prohibitively expensive because of the necessity of resolving a broad range of spatiotemporal scales. In turn, practitioners often rely on coarse-grained approximations of the original PDEs, trading off accuracy for reduced computational resources. To mitigate the loss of detail inherent in such approximations, closure models are employed to represent unresolved spatiotemporal interactions. We present a framework for developing closure models for PDEs using synthetic data acquired through the method of manufactured solutions. These data are used in conjunction with reinforcement learning to provide closures for coarse-grained PDEs. We illustrate the efficacy of our method using the one-dimensional and two-dimensional Burgers' equations and the two-dimensional advection equation. Moreover, we demonstrate that closure models trained for inhomogeneous PDEs can be effectively generalized to homogeneous PDEs. The results demonstrate the potential for developing accurate and computationally efficient closure models for systems with scarce data.

Problem

• Developing closure models for underresolved PDEs using synthetic data
• Improving coarse-grained PDE accuracy with reinforcement learning closures
• Generalizing closure models from inhomogeneous to homogeneous PDEs

Innovation

• Reinforcement learning for PDE closure models
• Synthetic data via manufactured solutions
• Generalizable to homogeneous PDEs