🤖 AI Summary
This work addresses the limitations of deterministic regression in settings involving coarse-graining, partial observability, or inverse problems, where input–output relationships are inherently one-to-many and conditional distributions exhibit irreducible stochasticity. To diagnose these challenges under finite data, the authors introduce a framework centered on the “conditional mean barrier,” proposing two diagnostic tools: a residual–feature orthogonality test and an upper-bound analysis of the coefficient of determination. These tools effectively disentangle model underfitting from irreducible conditional variance. Leveraging this diagnostic framework, the study systematically evaluates distribution-learning approaches—including negative log-likelihood, moment matching, variational objectives, adversarial divergences, and score matching—on benchmark problems such as bimodal distributions and multiscale Lorenz-96 closure tasks. Empirical results demonstrate that the framework clearly identifies the inadequacies of deterministic models and reveals the true variability of underlying conditional distributions.
📝 Abstract
Many problems in computational science and engineering become one-to-many after coarse graining, partial observation, or inverse reconstruction: a resolved state may not determine a unique subgrid forcing, a structural descriptor may not determine a unique effective response, and a low-resolution observation may correspond to many plausible high-resolution fields. In such settings, deterministic surrogates may learn a well-defined mathematical object while still missing application-relevant uncertainty. This tutorial develops a self-contained module centered on the conditional-mean barrier: the point at which a squared-loss predictor has reached the conditional mean and the remaining error is irreducible aleatoric variance. We give two diagnostics for locating this barrier, residual-feature orthogonality and the coefficient of determination against its explained-variance ceiling, and prove that adding latent randomness to a squared-loss predictor collapses it back to the conditional mean. Crossing the barrier therefore requires a loss that scores distributions rather than point predictions. We briefly organize common distributional objectives, including negative log-likelihood, moment and observable matching, variational objectives, adversarial divergences, and score matching, by the feature of the conditional law each targets. The emphasis is the boundary itself and a finite-data procedure for recognizing it, rather than a survey of methods beyond it. CPU-based demonstrations on a two-branch law and a two-scale Lorenz-96 closure problem show how the diagnostics distinguish deterministic underfitting from residual distributional variability.