Singular parameters and missing limits in neural PDE solvers

2026-10-05

AI Summary

This study addresses the inaccessibility of optimal solutions in neural PDE solvers, where unbounded parameter growth prevents finite models from representing the optimum. By revealing the intrinsic connection between missing limits and parameter divergence in Tanh networks, this work proposes a model space completion theory based on kernel derivatives. Specifically, it introduces kernel derivative operators to refine the function space structure. Integrating deep learning, numerical PDE methods, and function space completion theory, the authors prove that optimal approximation is attainable under standard assumptions. Numerical experiments validate both the identified parameter growth patterns and the effectiveness of the proposed completion strategy for PDE optimization, offering a novel paradigm for resolving the problem of unreachable optima.

Abstract

Neural solvers for partial differential equations (PDEs) can approach an accurate solution while their parameters grow without bound. In such cases, the limiting solution may have no finite representation in the chosen model, leaving the best loss unattained. Our analysis connects missing limits in deep neural tanh- networks to unbounded hidden parameters or increasingly redundant neurons. For a class of models built from translated kernels, we describe the missing functions and recover them by adding kernel derivatives to the model. This completion makes the best approximation attainable under standard assumptions. Numerical studies follow the associated parameter growth and explore how completion affects PDE optimization.

Problem

• Neural PDE solvers
• Singular parameters
• Missing limits
• Unbounded parameters
• Deep neural networks

Innovation

• Neural PDE solvers
• Missing limits
• Singular parameters
• Kernel derivatives
• Model completion