Geometric Calibration and Neutral Zones for Uncertainty-Aware Multi-Class Classification

2025-11-25

AI Summary

Neural network probability outputs in multiclass classification are often miscalibrated, and existing post-hoc calibration methods lack theoretical foundations. Method: We propose ALR (Additive Log-Ratio) calibration—a geometrically principled, information-geometric framework—extending ALR calibration to multiclass settings and naturally generalizing Platt scaling. By leveraging the Fisher–Rao metric on the probability simplex, we construct an interpretable calibration mapping and integrate a geometric reliability score with a distance-based neutral-zone decision rule to provably detect erroneous predictions. Contribution/Results: Theoretically, we establish calibration consistency and derive a concentration bound for reliability using M-estimation theory, sub-Gaussian inequalities, and the Bhattacharyya coefficient. Empirically, on a viral classification task, our two-stage method captures 72.5% of prediction errors at a 34.5% sample abstention rate, providing a statistically grounded, actionable mechanism for uncertainty-aware prediction.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Relational Probabilistic Models
• Computer Vision: Adversarial Attacks & Robustness

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

Modern artificial intelligence systems make critical decisions yet often fail silently when uncertain. We develop a geometric framework for post-hoc calibration of neural network probability outputs, treating probability vectors as points on the $(c-1)$-dimensional probability simplex equipped with the Fisher--Rao metric. Our approach yields Additive Log-Ratio (ALR) calibration maps that reduce exactly to Platt scaling for binary problems (Proposition~1) while extending naturally to multi-class settings -- providing a principled generalization that existing methods lack. Complementing calibration, we define geometric reliability scores based on Fisher--Rao distance and construct neutral zones for principled deferral of uncertain predictions. Theoretical contributions include: (i) consistency of the calibration estimator at rate $O_p(n^{-1/2})$ via M-estimation theory (Theorem~1), and (ii) tight concentration bounds for reliability scores with explicit sub-Gaussian parameters enabling sample size calculations for validation set design (Theorem~2). We conjecture Neyman--Pearson optimality of our neutral zone construction based on connections to Bhattacharyya coefficients. Empirical validation on Adeno-Associated Virus classification demonstrates that the two-stage framework (calibration followed by reliability-based deferral) captures 72.5% of errors while deferring 34.5% of samples. Notably, this operational gain is achievable with any well-calibrated probability output; the contribution of geometric calibration lies in its theoretical foundations rather than empirical superiority over simpler alternatives. This work bridges information geometry and statistical learning, offering formal guarantees relevant to applications requiring rigorous validation.

Problem

• Develop geometric calibration for neural network probability outputs
• Construct neutral zones for deferring uncertain predictions
• Bridge information geometry with statistical learning theory

Innovation

• Geometric calibration using Fisher-Rao metric on probability simplex
• ALR calibration maps generalizing Platt scaling to multi-class
• Neutral zones construction with geometric reliability scores for deferral