probabilistic spectral modelling

Traditional physicochemical soil analysis is costly and time-consuming, while spectral methods combined with machine learning, though efficient, often lack the predictive reliability required for laboratory-grade accuracy. This study proposes a “reject-and-retest” framework that introduces, for the first time in soil spectroscopy, a rejection mechanism based on prediction uncertainty. By leveraging tabular foundation models such as TabPFNv2.5 and TabICLv2 to quantify uncertainty from visible–near-infrared spectra, the approach automatically triggers conventional retesting for low-confidence predictions. Experiments on the Quebec Soil Spectral Library demonstrate that this method significantly reduces analytical costs while meeting user-specified accuracy thresholds, thereby enabling an AI-driven, adaptive measurement workflow that facilitates the integration of spectral techniques into routine laboratory practice.

Multivariate latent-variable regression models—such as Partial Least Squares (PLS), Principal Component Regression (PCR), and their kernelized variants—lack intrinsic uncertainty quantification, limiting their credibility in scientific modeling. This work introduces conformal inference to this class of models for the first time, proposing an input-dependent, data-adaptive prediction interval calibration framework that delivers finite-sample statistical guarantees without distributional assumptions. The method is theoretically rigorous, computationally efficient, and interpretable, and is applicable to complex regression tasks including near-infrared (NIR) spectroscopy and hyperspectral remote sensing. Empirical evaluation on synthetic data and real-world plant trait prediction demonstrates that the constructed 95% prediction intervals consistently achieve coverage rates close to the nominal level, significantly enhancing model reliability and robustness across diverse scenarios.

This study addresses the miscalibration of Gaussian processes (GPs) in uncertainty quantification (UQ)—specifically, their frequent lack of probabilistic calibration, which undermines convergence in downstream tasks such as Bayesian optimization. To tackle this, we propose Kernel Covariance Validation (KCV), the first framework to systematically exploit the multivariate normal structure of GP predictions for interpretable, computationally tractable calibration diagnostics. KCV integrates multivariate normality testing, uncertainty calibration analysis, and adaptive target design to quantitatively detect and localize model misspecification. Extensive experiments across 1D to high-dimensional GPs demonstrate KCV’s ability to identify canonical calibration failure modes. Results show that KCV significantly improves predictive reliability and enhances both the convergence speed and stability of optimization algorithms. By enabling principled, diagnostic-driven calibration, KCV establishes a new paradigm for trustworthy UQ in GP-based modeling.

Current spectral learning methods in scientific machine learning yield only point estimates, lacking uncertainty quantification—thus failing to meet trustworthiness requirements in safety-critical applications. To address this, we propose the Bayesian Parameterized Matrix Model (B-PMM), the first framework enabling Bayesian spectral uncertainty modeling under Hermitian constraints. Methodologically, B-PMM integrates adaptive spectral decomposition, regularization-aware matrix perturbation bounds, and manifold-aware Gaussian variational inference to ensure both geometric consistency and statistical reliability. Theoretically, we establish finite-sample calibration guarantees dependent on the spectral gap. Empirically, B-PMM achieves an expected calibration error (ECE) < 0.05 across matrices ranging from 5×5 to 500×500, demonstrating superior calibration and robust degradation behavior under spectral ill-conditioning.

Existing spectral response modeling for digital cameras is typically confined to isolated components, lacking an end-to-end, physically consistent description from illumination input to pixel intensity output—thus limiting color fidelity and spectral accuracy. This paper introduces the first full-chain, physics-driven end-to-end spectral–color joint modeling framework. It unifies the coupled effects of optical system transmission, sensor quantum efficiency, color filter array (CFA) spectral transmittance, and nonlinear pixel response. The model integrates empirically measured RGB camera spectral responses with data-driven nonlinear mapping correction. Evaluated under multiple illuminants, it achieves superior color reproduction (mean ΔE < 1.2) and significantly improved spectral reconstruction fidelity (37% reduction in RMSE). Validated across machine vision, remote sensing, and computational spectral imaging applications, this work bridges a critical theoretical and practical gap in end-to-end camera spectral response modeling.

This study addresses the challenge of efficiently and accurately predicting soil properties from high-dimensional, highly collinear spectral data across multiple spatial scales—from field to global. We systematically evaluate the performance of various regression models and dimensionality reduction techniques across 85 prediction tasks. Notably, we introduce TabPFN, a tabular foundation model, into soil spectroscopy for the first time, demonstrating its ability to outperform conventional approaches without explicit dimensionality reduction. Further gains in predictive accuracy are achieved by integrating partial least squares (PLS) latent variables with TabPFN. Empirical results show that TabPFN consistently achieves state-of-the-art performance across all scales, substantially surpassing classical baselines—particularly on large-scale global spectral libraries comprising tens of thousands of samples—thereby highlighting its potential as a general-purpose modeling paradigm for soil property prediction.

This study addresses key challenges in near-infrared (NIR) spectral calibration—including high-dimensional collinearity, limited sample sizes, reliance on preprocessing, sensitivity to outliers, and poor out-of-domain extrapolation—by introducing TabPFN, a tabular foundation model, to the field for the first time. The work proposes a novel paradigm enabling robust prediction without complex preprocessing. Within a unified evaluation framework across 66 NIR datasets, the authors systematically benchmark TabPFN against PLS/PLS-DA, Ridge regression, CatBoost, and 1D-CNN. Results demonstrate that, in regression tasks, TabPFN with optimized preprocessing achieves overall superior performance, while in classification tasks, TabPFN applied directly to raw spectra yields the best results—though its advantage diminishes in scenarios involving outliers or out-of-distribution extrapolation.

本文通过影响函数解决ESA的Ariel任务中机器学习模型的可解释性问题，提出了一种基于预测的影响函数方法，并利用该方法计算误差代理。

This study addresses the computational bottleneck in evaluating the calibration of nested uncertainty sets within expensive simulation models. To overcome this limitation, the work proposes an efficient calibration assessment method grounded in a Bayesian framework and the Dirichlet-Multinomial model. By exploiting the nested structure, the approach directly processes interval outputs without requiring access to the full predictive distribution. Furthermore, it incorporates Bayes factor testing for statistical inference, substantially reducing the number of independent simulations needed. The proposed method successfully detects model miscalibration in data assimilation tasks under limited simulation budgets. Overall, this work significantly lowers computational costs while demonstrating both the effectiveness and practical utility of the proposed approach for calibrating complex simulation systems.

This study addresses the instability, poor auditability, and sensitivity to small sample sizes inherent in traditional near-infrared spectroscopic modeling, which relies on external preprocessing searches. To overcome these limitations, the authors propose an Operator-Adaptive Calibration framework (AOM) that embeds common preprocessing techniques—such as Standard Normal Variate (SNV), Multiplicative Scatter Correction (MSC), and Asymmetric Least Squares (ASLS)—as learnable linear operators integrated within local ensemble branches, thereby preventing information leakage. This approach unifies preprocessing and model training while preserving the interpretability of wavelength coefficients. Built upon PLS and Ridge regression, AOM leverages efficient algorithms and dual kernel formulations to enable rapid training and exact coefficient recovery. Evaluated across more than 50 heterogeneous datasets, AOM-PLS achieves a median RMSEP reduction of 4% over conventional PLS (outperforming it in 42 cases), while AOM-Ridge yields an average improvement of 2.22% (winning in 35 cases), with training times of only a few seconds.