Into the danger zone: stable extrapolation in high-dimensional function and operator learning

2026-09-29

AI Summary

This study addresses the challenge of out-of-distribution generalization in scientific machine learning by investigating the theoretical limits of stable extrapolation for high-dimensional functions and operators. Leveraging polynomial, deep neural network, and neural operator frameworks combined with complex analysis and high-dimensional statistics, this work reveals a "blessing of dimensionality" phenomenon: it demonstrates that higher-order coordinate smoothness ensures algebraic error convergence, thereby overcoming the pessimistic bounds of conventional theory. The core contribution lies in deriving explicit convergence rates under arbitrary test measures, establishing extrapolation guarantees that depend solely on the support domain rather than the specific test distribution. Numerical experiments validate the effectiveness of these theoretical findings across extensive extrapolation regimes.

Abstract

Out-of-distribution (OOD) generalization is a central challenge in scientific machine learning. We study regression problems in which the test distribution differs from the training distribution and ask: under what assumptions on the target function or operator is stable extrapolation possible, and how far beyond the training domain can one extrapolate? Existing theory controls the test error through additive penalties measuring the discrepancy between the training and test distributions. Such guarantees show robustness to small distribution shifts, but can very pessimistic in comparison to OOD performance observed empirically. We identify classes of holomorphic functions and operators for which the OOD generalization error converges at algebraic rates even in the presence of large distribution shifts. This phenomenon stems from the increasing smoothness of higher-index coordinates, leading to what we term a `blessing of high dimensionality'. For learning with either polynomials, deep neural networks or deep neural operators, we derive explicit rates for arbitrary test measures supported on suitable domains and quantify how the admissible domain depends on the underlying regularity of the function or operator. Our extrapolation guarantees are independent of the test distribution, depending only on its support. We also present a series of numerical experiments across a range of functions and operators that support the main theoretical findings.

Problem

• out-of-distribution generalization
• extrapolation
• scientific machine learning
• high-dimensional function learning
• operator learning

Innovation

• Out-of-distribution generalization
• Holomorphic functions and operators
• Blessing of high dimensionality
• Deep neural operators
• Stable extrapolation