perturbation methods

Machine learning training is significantly more time-consuming than inference, and the design of input or parameter perturbations has long relied on empirical trial-and-error. Method: This paper models training dynamics as a first-passage process and introduces a statistical mechanics framework to analyze model responses to input/parameter perturbations. It proposes, for the first time, a single-frequency perturbation response theory grounded in the quasi-stationary assumption, and rigorously proves its generalizability to multi-frequency perturbation regimes—enabling rational optimization of perturbation protocols. Contribution/Results: Evaluated on ResNet-18 trained for CIFAR-10 classification, the method precisely identifies the optimal perturbation type and frequency, reducing training iterations by 23% and improving test accuracy by 1.4 percentage points, thereby substantially enhancing both training efficiency and generalization performance.

This study addresses accuracy and consistency issues in the numerical construction of the Karhunen–Loève expansion (KLE) arising from discretization, quadrature rules, and finite sample sizes. It establishes an algebraic equivalence between the spectral decomposition of the Fredholm integral equation and the singular value decomposition (SVD) of a weighted sample covariance matrix, thereby unifying model-driven and data-driven KLE frameworks. The work innovatively constructs the covariance function on a non-simply-connected three-dimensional toroidal domain using the shortest interior path distance, and implements the approach numerically with unstructured meshes and Gaussian quadrature. Experiments demonstrate that, in a one-dimensional benchmark problem, SVD-based eigenvalue estimates and empirical KL coefficients converge to the theoretical 𝒩(0,1) distribution. In two-dimensional irregular and three-dimensional toroidal domains, the study systematically quantifies the combined influence of discretization strategy, quadrature accuracy, and sample size on KLE reconstruction error.

This work addresses efficient solving of parametric partial differential equations (PDEs) and Bayesian inverse problems. We systematically investigate and compare three neural operator architectures—DeepONet, PCANet, and the Fourier Neural Operator (FNO)—and propose a residual error-driven correction mechanism coupled with multi-level collaborative training to significantly enhance generalization and accuracy in both forward prediction and inverse posterior inference. To our knowledge, this is the first unified end-to-end framework for neural operators applied to both PDE forward and inverse problems. We validate the approach on Poisson equation and linear elastic deformation tasks: Bayesian posterior sampling accelerates over two orders of magnitude relative to conventional MCMC methods while preserving high-fidelity solution accuracy. The study establishes a reproducible, scalable technical pathway for engineering deployment of neural operators in scientific computing.

Data-driven modeling of nonlinear time-varying partial differential equations (PDEs) suffers from prohibitive computational cost in generating training samples—conventional numerical solvers require thousands of time steps, vastly exceeding typical model training iterations. Method: We propose Homologous Perturbation Sample Synthesis (HOPSS), the first approach to introduce homologous perturbation into PDE data generation. HOPSS leverages a small set of high-fidelity base solutions computed via an accurate solver, then efficiently synthesizes high-quality training samples through temporal downsampling, controlled stochastic noise injection, and inter-solution differencing to construct accurate right-hand-side (RHS) terms. Results: On the Navier–Stokes equations, HOPSS generates 10,000 training samples using only 10% of the computational time required by conventional solvers, while preserving model accuracy. Its core contribution is a paradigm shift from expensive long-time numerical integration to high-fidelity, low-overhead data augmentation.

This work addresses the inefficiency of conventional neural spectral methods for time-dependent partial differential equations, which require learning the entire vector field. The authors propose a residual decomposition framework that splits the solution into a low-fidelity background field and a high-resolution perturbation, modeling only the unresolved dynamics. Their approach integrates a fixed spectral operator, a background-dependent correction term, and an optional neural closure component. Crucially, the neural closure is formulated as a conditional refinement dependent on background fidelity, residual structure, and model compatibility, thereby explicitly separating physical mechanisms from neural components. On 2D Burgers, Klein-Gordon, and heterogeneous wave equations, their deterministic solver surpasses the NeuSA baseline without any training; for the Burgers equation, it achieves 24-fold and 44-fold reductions in training and extrapolation errors, respectively.

This study addresses the ill-posedness of integral equations arising from the curse of dimensionality in high-dimensional nonparametric instrumental variable (NPIV) estimation. For the first time, it introduces physical perturbation theory to this domain and proposes a kernel ridge regression–based higher-order perturbation correction method. By leveraging spectral analysis of integral operators and hybridization of eigenmodes, the approach systematically constructs a perturbation expansion that effectively mitigates high-dimensional ill-posedness while preserving computational tractability. Theoretical analysis and empirical experiments demonstrate that, with first-order correction, the method reduces prediction error by up to 99% under severe ill-posedness (β > 0.7), with performance gains intensifying as dimensionality increases—substantially outperforming existing approaches.

This study investigates the hierarchical learning mechanism of infinitely wide two-layer neural networks when trained on misspecified single-index models in high dimensions. By introducing a perturbation parameter to modulate the disparity in training speeds between the two layers, and combining quantitative approximation theory for singularly perturbed flows near an integral-constraint manifold with dynamic analysis of the empirical weight measure, the work rigorously establishes—for the first time—that constant and linear components can be precisely recovered within the prediction-relevant timescale. It further reveals their essential role in enabling subsequent learning of quadratic components. The analysis explicitly identifies the thresholds and timescales governing this hierarchy and demonstrates that, during quadratic component learning, only a sparse subset of neurons exhibits significant growth, while the remainder reorganize to preserve the already-learned structure.

This work proposes a physics-informed neural network method based on a Petrov–Galerkin variational framework for two-dimensional singularly perturbed problems involving one or two small perturbation parameters. It is the first to integrate this variational framework with neural networks to efficiently resolve sharp boundary layers and multiscale features. The approach constructs a neural network trial solution space, employs tensor-product hat functions as test functions, computes source terms via automatic differentiation, and enforces Dirichlet boundary conditions strongly. Numerical experiments on benchmark problems demonstrate high accuracy in both the maximum norm and the $L^2$ norm, confirming the method’s effectiveness, robustness, and precise resolution of boundary layers in multiscale modeling.

This work addresses the sensitivity to initial guesses and high computational cost of Newton’s method for solving nonlinear parameterized partial differential equations. The authors propose a two-stage initialization strategy: first, by leveraging parameter sampling and a precomputed solution library, they construct two complementary feature spaces—solution manifold and corrected search directions—from discrete Newton trajectories; second, a regression model predicts a surrogate initial guess, which is then refined via lightweight GMRES-based residual minimization to yield a high-quality starting point. Operating under a weakly intrusive framework, this approach significantly accelerates high-fidelity Newton iterations, markedly reducing both iteration counts and total CPU time on benchmark PDE problems, outperforming existing methods that rely solely on surrogate-based initialization.