Residual Correlation as a Diagnostic for Joint-Uncertainty Gains from GP Coregionalisation

2026-09-24

AI Summary

This study addresses the instability of collaborative Gaussian process (GP) benefits and the difficulty in determining optimal coupling timing in multi-target regression. We propose a residual correlation-based diagnostic metric to quantify joint uncertainty gains and design a Residual-ICM model to optimize the covariance structure. Experimental validation integrates intrinsic-model collaborative GPs, independent GPs, and Transformer/CNN feature representations. Results demonstrate that while point prediction performance remains comparable, the proposed diagnostic metric exhibits a strong correlation with improvements in negative log-likelihood (NLL) (ρ = −0.83). Furthermore, Residual-ICM achieves the best joint NLL. This work provides a reliable criterion for identifying effective coupling strategies in multi-task learning, enabling more principled deployment of collaborative GP frameworks without compromising predictive accuracy.

Abstract

In multi-target regression, correlated targets are often coupled through multi-output Gaussian processes with an intrinsic model of coregionalisation (GP-ICM), assuming that sharing statistical strength improves overall performance. In practice, the benefits are inconsistent. Across the settings studied, we find that the main benefit of coregionalisation is joint uncertainty quantification rather than point prediction. Raw target correlation does not predict when coupling helps; in the separable GP-ICM settings studied here, residual correlation, the cross-target dependence left unexplained by independent per-target predictors, is the strongest predictor of joint-uncertainty gains. We introduce a lightweight diagnostic, $D_{\rm logdet}=-\frac{1}{2}\log\det R_{\rm res}$, which represents the idealised joint negative log-likelihood (NLL) gain from modelling a full rather than diagonal residual covariance and is computable from independent GPs alone. Across a controlled synthetic study, 16 multi-target benchmarks, and frozen transformer and convolutional neural network representations for keypoint regression, point prediction remains largely unchanged ($ΔR^2\approx 0$). In contrast, $D_{\rm logdet}$ strongly predicts observed ICM NLL improvements ($ρ_s=-0.83$, $p<0.001$), outperforming heuristics such as the feature-to-sample ratio. We also propose Residual-ICM, which preserves independent marginal variances while adding residual-correlation structure to the joint covariance. Residual-ICM achieves the best average joint NLL among the compared methods, while the diagnostic indicates when covariance coupling is likely to be useful. The diagnostic is specific to global Gaussian residual dependence, the structure captured by separable coregionalisation.

Problem

• multi-target regression
• Gaussian process coregionalisation
• joint uncertainty quantification
• residual correlation

Innovation

• Multi-output Gaussian Processes
• Coregionalisation
• Joint Uncertainty Quantification
• Residual Correlation
• Residual-ICM