R^2-HGP: A Double-Regularized Gaussian Process for Heterogeneous Transfer Learning

2025-12-10

AI Summary

In heterogeneous transfer learning, multi-output Gaussian processes (MGP) face three key challenges: input-space mismatch between source and target domains, lack of integration of physical priors, and risk of negative transfer. To address these, this paper proposes an end-to-end trainable dual-regularized Gaussian process framework. Our contributions are threefold: (1) a learnable input alignment mapping explicitly models heterogeneous input relationships across domains; (2) physics-informed constraints are embedded into a conditional variational autoencoder (CVAE) architecture, serving as a prior regularization term to ensure stable and physically consistent mapping; and (3) a sparse transfer coefficient mechanism adaptively selects beneficial source tasks to mitigate negative transfer. Evaluated on both synthetic benchmarks and real-world engineering cases, the method consistently outperforms state-of-the-art approaches in prediction accuracy, uncertainty calibration, and transfer robustness—demonstrating strong effectiveness and generalizability.

Technology Category

• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Search and Optimization: Learning to Search
• Constraint Satisfaction and Optimization: Constraint Learning and Acquisition

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs

Abstract

Multi-output Gaussian process (MGP) models have attracted significant attention for their flexibility and uncertainty-quantification capabilities, and have been widely adopted in multi-source transfer learning scenarios due to their ability to capture inter-task correlations. However, they still face several challenges in transfer learning. First, the input spaces of the source and target domains are often heterogeneous, which makes direct knowledge transfer difficult. Second, potential prior knowledge and physical information are typically ignored during heterogeneous transfer, hampering the utilization of domain-specific insights and leading to unstable mappings. Third, inappropriate information sharing among target and sources can easily lead to negative transfer. Traditional models fail to address these issues in a unified way. To overcome these limitations, this paper proposes a Double-Regularized Heterogeneous Gaussian Process framework (R^2-HGP). Specifically, a trainable prior probability mapping model is first proposed to align the heterogeneous input domains. The resulting aligned inputs are treated as latent variables, upon which a multi-source transfer GP model is constructed and the entire structure is integrated into a novel conditional variational autoencoder (CVAE) based framework. Physical insights is further incorporated as a regularization term to ensure that the alignment results adhere to known physical knowledge. Next, within the multi-source transfer GP model, a sparsity penalty is imposed on the transfer coefficients, enabling the model to adaptively select the most informative source outputs and suppress negative transfer. Extensive simulations and real-world engineering case studies validate the effectiveness of our R^2-HGP, demonstrating consistent superiority over state-of-the-art benchmarks across diverse evaluation metrics.

Problem

• Aligns heterogeneous input domains for effective knowledge transfer
• Incorporates physical insights as regularization to ensure stable mappings
• Adaptively selects informative sources to prevent negative transfer

Innovation

• Trainable prior mapping aligns heterogeneous input domains
• Conditional variational autoencoder integrates multi-source transfer GP
• Sparsity penalty on transfer coefficients prevents negative transfer