orthogonalized estimation

In Gaussian process regression (GPR), active learning lacks theoretical guarantees on prediction accuracy due to the unknown underlying target distribution. Method: This paper introduces the first distributionally robust active learning framework for GPR, proposing two sampling strategies that minimize the worst-case expected mean squared error (WCE-MSE). Theoretically, we derive a tight upper bound on WCE-MSE and prove that it can be made arbitrarily small under finite labeled samples. Computationally, we integrate distributionally robust optimization with kernel ridge regression approximation to ensure tractability and scalability. Results: Extensive experiments on synthetic and real-world datasets demonstrate that our methods significantly outperform classical active learning baselines. Crucially, they achieve substantial improvements in both labeling efficiency and generalization robustness—each rigorously supported by theoretical guarantees.

This work addresses the challenge of achieving both interpretability and high accuracy in nonlinear system identification when physical models are incomplete. The authors propose a semi-parametric modeling framework that, for the first time, enables orthogonal decoupling between physics-based white-box components and data-driven bias terms. By employing orthogonal Gaussian process regression, the method jointly optimizes sparse physical parameter selection and black-box bias learning. This approach preserves model interpretability while significantly enhancing predictive accuracy, thereby establishing a high-fidelity, interpretable nonlinear system identification model suitable for scenarios where only partial physical knowledge is available.

Existing Gaussian process regression (GPR) uncertainty quantification methods rely on fixed input locations, posterior variance scaling, or hyperparameter tuning, limiting their ability to characterize global extremal behavior and yielding poorly robust upper/lower bounds on unseen data. This work proposes the first chain-structured GPR uncertainty quantification framework that requires neither prespecified input points nor variance scaling. By leveraging kernel-specific analysis (e.g., RBF, Matérn) and partition-diameter-driven local geometric modeling, it delivers globally valid extremal bounds and adaptive local uncertainty measures. Theoretically, its bound tightness surpasses that of analytic relaxation approaches. Empirical evaluation on synthetic and real-world benchmarks demonstrates significant improvements over state-of-the-art baselines in bound tightness, robustness to distributional shift, and generalization across diverse tasks.

Gaussian process regression (GPR) is often treated as a black-box surrogate model, limiting its pedagogical utility and interpretability in uncertainty quantification (UQ) for beginners. Core UQ tasks—including uncertainty propagation, risk estimation, Bayesian optimization, parameter inference, and sensitivity analysis—require deeper engagement with GPR’s inherent probabilistic structure. Method: This work develops a systematic, pedagogically grounded GPR-based UQ framework that integrates UQ-specific techniques: Bayesian quadrature, active learning, and surrogate-based sensitivity analysis. It emphasizes principled covariance kernel design, Bayesian hyperparameter estimation, and reproducible implementation. Contribution/Results: The framework lowers the barrier to applying GPR in complex UQ scenarios, enhances model transparency and decision reliability, and provides a theoretically rigorous yet practically actionable paradigm for UQ education and research across engineering and scientific disciplines.

Gaussian processes (GPs) struggle to rigorously incorporate uncountably infinite-dimensional functional prior information—such as boundary conditions or global physical constraints satisfied by PDE solutions. Method: This paper proposes a unified modeling framework grounded in reproducing kernel Hilbert spaces (RKHS), establishing for the first time a rigorous equivalence between the GP conditional expectation and orthogonal projection in RKHS. This enables direct embedding of functional constraints (e.g., Dirichlet or Neumann boundary conditions) into the GP prior, bypassing conventional pseudo-point approximations. Contribution/Results: We provide theoretical guarantees on existence, uniqueness, and convergence of the constrained GP posterior. Computationally, we design a practical numerical approximation algorithm. Experiments on PDE inverse problems demonstrate substantial improvements in uncertainty quantification accuracy and posterior consistency. The framework delivers a rigorous, general, and computationally tractable paradigm for integrating domain knowledge into Bayesian modeling.

This study addresses the vulnerability of traditional Gaussian Process Regression (GPR) to outliers caused by Gaussian likelihood assumptions by proposing a Generative Gaussian Process Regression model. By constructing an observation-level contamination generative model and employing a variational generalized expectation-maximization algorithm, this approach enables adaptive identification and suppression of outliers. Experiments on both synthetic and real-world datasets demonstrate that the proposed model achieves predictive accuracy superior to or comparable with existing robust GPR methods while maintaining cubic computational complexity. Consequently, this work effectively balances robustness with computational efficiency, offering a novel paradigm for Bayesian regression in noisy environments.

Traditional Gaussian process regression often yields physically inconsistent results when reconstructing full-field modal shapes from sparse sensor data. This work proposes a physics-constrained single-output Gaussian process framework (CONS-SOGP), which, for the first time, embeds mass orthogonality constraints directly into Gaussian process regression. By jointly optimizing independent modal kernels and an orthogonality penalty term, the method achieves high-fidelity modal expansion while preserving physical plausibility. Leveraging marginal likelihood derivation and gradient-based hyperparameter optimization, CONS-SOGP demonstrates significantly improved performance over existing Gaussian process approaches in numerical experiments on multi-degree-of-freedom structures, yielding more accurate and reliable reconstructions of modal shapes.

论文指出高斯核函数因过于平滑导致预测方差过小及数值不稳定问题，建议避免使用，并提出非解析核函数作为替代。

This work addresses the lack of theoretical guarantees for nonparametric regression in reproducing kernel Hilbert spaces under model misspecification, high-dimensional settings, and nonconvex losses. It establishes a unified theoretical framework for regularized M-estimators encompassing a broad class of both convex and nonconvex loss functions. By introducing a novel complexity measure, the analysis achieves an explicit bias–variance decomposition. Leveraging tools from functional analysis and empirical process theory, the study proves the existence, measurability, and asymptotic linearity of the estimator without requiring closed-form solutions or global Lipschitz assumptions. Notably, within tensor-product Sobolev spaces, the framework reveals a mechanism to circumvent the curse of dimensionality, yielding minimax-optimal convergence rates that depend on mixed smoothness of the underlying function. The variance component is shown to be robust to model misspecification, and numerical experiments in C++ corroborate the theoretical findings.

This work addresses the challenge of incorporating prior features into kernel methods without penalizing them, thereby enhancing regression performance. To this end, the authors propose Conditional Kernel Ridge Regression (Conditional KRR), which decomposes the target function into a prior component modeled within a prescribed function class and a residual component, applying kernel regularization only to the latter. Theoretical analysis reveals that this approach is equivalent to standard Kernel Ridge Regression augmented with a controllable error term, and it achieves improved statistical risk under settings such as principal components or random features. When the prior component dominates the target function, Conditional KRR substantially outperforms standard KRR, a finding corroborated by both theoretical guarantees and empirical experiments.