Model Error Embedding with Orthogonal Gaussian Processes

2026-02-20

AI Summary

This work proposes an embedded model error correction framework to address prediction biases arising from simplifying assumptions in computational models of physical systems, which often introduce errors entangled with model parameters. The approach characterizes the spatiotemporal correlations of model discrepancy using a weight-space representation of Gaussian processes and incorporates orthogonality constraints tailored for nonlinear systems to effectively decouple model and error parameters. By integrating a likelihood-informed subspace method, the framework efficiently handles high-dimensional problems. Under Bayesian inference, the method successfully corrects predictions to align with observed trends in both linear and nonlinear test cases, reverts to the prior predictive distribution during extrapolation, and yields nearly uncorrelated posteriors for model and error parameters—thereby substantially enhancing model reliability and interpretability.

Technology Category

• Reasoning under Uncertainty: Relational Probabilistic Models
• Machine Learning: Calibration & Uncertainty Quantification
• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning

Application Category

• User Modeling, Personalization and Recommendation: Psychology-informed user models and recommender systems
• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Computational models of complex physical systems often rely on simplifying assumptions which inevitably introduce model error, with consequent predictive errors. Given data on model observables, the estimation of parameterized model-error representations, along with other model parameters, would be ideally done while separating the contributions of each of the two sets of parameters, in order to ensure meaningful stand-alone model predictions. This work builds an embedded model error framework using a weight-space representation of Gaussian processes (GPs) to flexibly capture model-error spatiotemporal correlations and enable inference with GP-embedding in non-linear models. To disambiguate model and model-error/bias parameters, we extend an existing orthogonal GP method to the embedded model-error setting and derive appropriate orthogonality constraints. To address the increased dimensionality introduced by the GP representation, we employ the likelihood-informed subspace method. The construction is demonstrated on linear and non-linear examples, where it effectively corrects model predictions to match data trends. Extrapolation beyond the training data recovers the prior predictive distribution, and the orthogonality constraints lead to meaningful stand-alone model predictions and nearly uncorrelated posteriors between model and model-error parameters.

Problem

• model error
• parameter disambiguation
• Gaussian processes
• orthogonality constraints
• predictive accuracy

Innovation

• Orthogonal Gaussian Processes
• Model Error Embedding
• Weight-space Representation
• Likelihood-informed Subspace
• Parameter Disambiguation