🤖 AI Summary
This study addresses the need for a unified understanding of the relationships and generalization capabilities across data-driven modeling paradigms, ranging from classical inverse problems to modern neural operators. By integrating inverse problem theory, sparse identification of dynamical systems, neural ordinary differential equations, and neural operators—and further incorporating the philosophical notion of “mechanism” from philosophy of science—the authors construct a cohesive analytical framework. The work demonstrates that genuine mechanistic discovery and robust cross-scenario generalization are achievable only when models recover concise differential equation structures underlying the observed data. This perspective clarifies the fundamental connections among diverse modeling approaches and underscores the critical role of mechanistic interpretability in enabling reliable generalization, thereby offering a theoretical foundation for the categorization, selection, and design of scientific machine learning models.
📝 Abstract
Scientists have historically relied on mathematical models based on differential equations to relate system inputs -- forces, fluxes, or heat sources -- to outputs, such as displacement, velocity, concentration, and temperature. These models rely on deep domain knowledge to determine the form of the governing differential equation, which is then calibrated with data by solving an inverse problem. In recent years, the field of Scientific Machine Learning has introduced a variety of alternative modeling strategies for physical systems. A method called Sparse Identification of Nonlinear Dynamics learns the governing equation as a sparse linear combination of terms in a user-defined library. Neural Ordinary Differential Equations construct the governing equation by taking in the state and its derivatives at the input layer of a neural network. Entirely foregoing the modeling framework of differential equations, neural operators directly learn a non-linear mapping between the system inputs and outputs. From inverse problems to neural operators, all of these modeling strategies can be conceptualized as data-driven machinery to predict a system's response over a range of inputs. It is then natural to wonder how exactly these various strategies relate to each other, and whether they can be neatly taxonomized. Drawing from the philosophical literature on scientific models, we argue that many model types have a common structure, differing only in the assumed model class of the input-output relation they define. Connecting to philosophical ideas on mechanism, and arguing that data from physical systems arises from solutions to parsimonious differential equations, we propose that only certain models are capable of mechanism discovery, and thus generalization. Our analysis is intended to unite apparently disparate modeling strategies and provide insight into their appropriate use cases.