🤖 AI Summary
Estimating predictive uncertainty for quantized neural networks in safety-critical applications remains challenging, particularly when model parameters or gradients are inaccessible.
Method: We propose a post-hoc uncertainty calibration method that requires only the backbone network’s raw inputs and its frozen outputs—no parameter access or gradient computation. For regression tasks, we formulate an input–frozen-output joint conditional framework to directly infer Gaussian posterior distribution parameters. We introduce the first formally falsifiable posterior optimization objective, rigorously characterize conditions for output-driven uncertainty estimation validity, and theoretically prove that frozen outputs encode generalization error information. Using maximum likelihood estimation and sequential fitting, we train lightweight regression surrogates on data-augmented subsets.
Results: Our method significantly improves out-of-distribution detection and probabilistic calibration on UCI and deep regression benchmarks, preserves input-dependent uncertainty modeling, incurs zero sampling or approximation overhead during inference, and maintains the backbone’s original prediction accuracy.
📝 Abstract
Uncertainty quantification is critical in safety-sensitive applications but is often omitted from off-the-shelf neural networks due to adverse effects on predictive performance. Retrofitting uncertainty estimates post-hoc typically requires access to model parameters or gradients, limiting feasibility in practice. We propose a theoretically grounded framework for post-hoc uncertainty estimation in regression tasks by fitting an auxiliary model to both original inputs and frozen model outputs. Drawing from principles of maximum likelihood estimation and sequential parameter fitting, we formalize an exact post-hoc optimization objective that recovers the canonical MLE of Gaussian parameters, without requiring sampling or approximation at inference. While prior work has used model outputs to estimate uncertainty, we explicitly characterize the conditions under which this is valid and demonstrate the extent to which structured outputs can support quasi-epistemic inference. We find that using diverse auxiliary data, such as augmented subsets of the original training data, significantly enhances OOD detection and metric performance. Our hypothesis that frozen model outputs contain generalizable latent information about model error and predictive uncertainty is tested and confirmed. Finally, we ensure that our method maintains proper estimation of input-dependent uncertainty without relying exclusively on base model forecasts. These findings are demonstrated in toy problems and adapted to both UCI and depth regression benchmarks. Code: https://github.com/biggzlar/IO-CUE.