Efficient multi-fidelity Gaussian process regression for noisy outputs and non-nested experimental designs

2025-11-25

AI Summary

This paper addresses Gaussian process regression modeling with non-nested, noisy multi-fidelity data. We propose an efficient, scalable multi-fidelity surrogate model that abandons the conventional recursive autoregressive assumption and instead introduces parameterized linear predictors, for which we derive closed-form update rules and integrate an expectation-maximization (EM) algorithm to estimate high-fidelity hyperparameters. A decoupled optimization strategy is further designed to substantially reduce computational complexity. Our key contributions are threefold: (i) the first method supporting simultaneous non-nested data structures and observation noise in multi-source fusion; (ii) a theoretically grounded analytical framework for learning closed-form solutions; and (iii) consistent superiority over state-of-the-art approaches in both prediction accuracy and training efficiency across multiple benchmarks and real-world tasks, with systematic experiments confirming strong generalizability and scalability.

Technology Category

• Machine Learning: Multi-instance/Multi-view Learning
• Search and Optimization: Non-convex Optimization
• Natural Language Processing: Learning & Optimization for NLP

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

This paper presents a multi-fidelity Gaussian process surrogate modeling that generalizes the recursive formulation of the auto-regressive model when the high-fidelity and low-fidelity data sets are noisy and not necessarily nested. The estimation of high-fidelity parameters by the EM (expectation-maximization) algorithm is shown to be still possible in this context and a closed-form update formula is derived when the scaling factor is a parametric linear predictor function. This yields a decoupled optimization strategy for the parameter selection that is more efficient and scalable than the direct maximum likelihood maximization. The proposed approach is compared to other multi-fidelity models, and benchmarks for different application cases of increasing complexity are provided.

Problem

• Generalizes multi-fidelity Gaussian process for noisy non-nested data
• Enables efficient parameter estimation via EM algorithm optimization
• Provides scalable surrogate modeling for complex multi-fidelity applications

Innovation

• Multi-fidelity Gaussian process for noisy non-nested data
• EM algorithm with closed-form parametric scaling updates
• Decoupled optimization strategy for efficient parameter selection