🤖 AI Summary
To address the growing demand for trustworthy AI, this work proposes a calibratable nonparametric probabilistic regression tree that directly models the conditional cumulative distribution function (cCDF) to produce prediction intervals with high accuracy, reliable coverage, and statistical calibration. Methodologically, it introduces efficient tree-splitting algorithms tailored to the weighted interval score (WIS) and continuous ranked probability score (CRPS)—the first such formulations for these proper scoring rules. To accelerate split search and gradient updates, it synergistically integrates min-max heaps, weight-balanced binary trees, and Fenwick trees. Additionally, the framework supports group-conditional conformal prediction, ensuring statistical validity while preserving model interpretability. Empirical evaluation demonstrates superior calibration, coverage, and computational efficiency compared to state-of-the-art baselines across diverse benchmarks. The method achieves a balanced advantage in critical scientific and engineering applications where both uncertainty quantification rigor and practical deployability are essential.
📝 Abstract
The perspective of developing trustworthy AI for critical applications in science and engineering requires machine learning techniques that are capable of estimating their own uncertainty. In the context of regression, instead of estimating a conditional mean, this can be achieved by producing a predictive interval for the output, or to even learn a model of the conditional probability $p(y|x)$ of an output $y$ given input features $x$. While this can be done under parametric assumptions with, e.g. generalized linear model, these are typically too strong, and non-parametric models offer flexible alternatives. In particular, for scalar outputs, learning directly a model of the conditional cumulative distribution function of $y$ given $x$ can lead to more precise probabilistic estimates, and the use of proper scoring rules such as the weighted interval score (WIS) and the continuous ranked probability score (CRPS) lead to better coverage and calibration properties. This paper introduces novel algorithms for learning probabilistic regression trees for the WIS or CRPS loss functions. These algorithms are made computationally efficient thanks to an appropriate use of known data structures - namely min-max heaps, weight-balanced binary trees and Fenwick trees. Through numerical experiments, we demonstrate that the performance of our methods is competitive with alternative approaches. Additionally, our methods benefit from the inherent interpretability and explainability of trees. As a by-product, we show how our trees can be used in the context of conformal prediction and explain why they are particularly well-suited for achieving group-conditional coverage guarantees.