🤖 AI Summary
This work addresses the lack of systematic benchmarks and evaluation frameworks for inverse problems involving partial differential equations (PDEs) by introducing PDEInvBench, the first comprehensive benchmark dataset encompassing numerical simulations of diverse time-varying and time-invariant PDEs. The study systematically explores the neural network design space across three dimensions—optimization strategies, problem representations, and model/data scale—and evaluates architectures with varying inductive biases and conditioning strategies through supervised learning, self-supervised learning, and test-time optimization. Key findings reveal that a two-stage training protocol substantially enhances performance, incorporating PDE derivative features as inputs consistently improves accuracy, and increasing the diversity of initial conditions yields greater gains than merely expanding the parameter range.
📝 Abstract
Inverse problems in partial differential equations (PDEs) involve estimating the physical parameters of a system from observed spatiotemporal solution fields.Neural networks are well-suited for PDE parameter estimation due to their capability to model function-to-function space transformations. While existing benchmarks of machine learning methods for PDEs primarily focus on the forward problem, there are no similar comprehensive studies and benchmark datasets on PDE inverse problems, i.e., mapping solution fields to underlying physical parameters. We fill this gap by introducing PDEInvBench, a comprehensive benchmark dataset consisting of numerical simulations for both time-dependent and time-independent PDEs across a wide range of physical behaviors and parameters. Our dataset includes evaluation splits that assess performance in both in-distribution and various out-of-distribution settings. Using our benchmark dataset, we comprehensively explore the design space of neural networks for PDE inverse problems along three key dimensions: (1) optimization procedures, analyzing the role of supervised, self-supervised, and test-time training objectives on performance, (2) problem representations, where we study the value of architectural choices with different inductive biases and various conditioning strategies, and (3) scaling, which we perform with respect to both model and data size. Our experiments reveal several practical insights: 1) neural networks perform best with a two-stage training procedure: initial supervision with PDE parameters followed by test-time fine-tuning using the PDE residual, 2) incorporating PDE derivatives as input features consistently improves accuracy, and 3) increasing the diversity of initial conditions in the training data yields greater performance gains than expanding the range of PDE parameters. We make our dataset and codebase publicly available.