Optimization and generalization analysis for two-layer physics-informed neural networks without over-parametrization

2025-07-22

AI Summary

Physics-informed neural networks (PINNs) suffer from high computational cost and a theory-practice gap due to excessive overparameterization. Method: We investigate the optimization and generalization of two-layer PINNs in the *non-overparameterized* regime. We derive a width threshold—dependent solely on the target accuracy ε and intrinsic problem constants—that avoids reliance on unrealistically large architectures. Within a least-squares regression framework, we conduct the first joint convergence analysis for non-overparameterized PINNs under stochastic gradient descent (SGD), assuming mild regularity of the target function. Contribution/Results: We prove that, when network width exceeds this threshold, both training loss and expected generalization error are bounded by O(ε). This result bridges theoretical guarantees with practical training behavior, significantly enhancing interpretability and consistency. Our analysis establishes a new paradigm for designing efficient, verifiable PINNs without artificial overparameterization.

Technology Category

• Search and Optimization: Non-convex Optimization
• Machine Learning: Optimization
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Social Networks and Social Media: Influence propagation, information diffusion, and the prediction on networks

Abstract

This work focuses on the behavior of stochastic gradient descent (SGD) in solving least-squares regression with physics-informed neural networks (PINNs). Past work on this topic has been based on the over-parameterization regime, whose convergence may require the network width to increase vastly with the number of training samples. So, the theory derived from over-parameterization may incur prohibitive computational costs and is far from practical experiments. We perform new optimization and generalization analysis for SGD in training two-layer PINNs, making certain assumptions about the target function to avoid over-parameterization. Given $ε>0$, we show that if the network width exceeds a threshold that depends only on $ε$ and the problem, then the training loss and expected loss will decrease below $O(ε)$.

Problem

• Analyzes SGD for PINNs without over-parametrization
• Avoids high computational costs from wide networks
• Proves loss bounds below O(ε) with minimal width

Innovation

• Avoids over-parametrization in PINNs
• Uses two-layer neural networks
• Analyzes SGD optimization and generalization