Hierarchical Utility Calibration for Structured Multiclass Decisions

2026-09-28

AI Summary

This study addresses the limitation of conventional utility calibration in multi-level label classification, where node-wise errors mutually cancel out and thereby obscure local biases. We are the first to formally reveal this cancellation mechanism and propose Hierarchical Utility Calibration (HUC). Specifically, HUC employs a hierarchical utility decomposition algorithm to precisely attribute global errors to individual internal nodes, achieving node-wise calibration through finite-sample interval estimation combined with an HUC-Boost directed update strategy. The proposed method significantly improves calibration accuracy across all hierarchy levels while preserving target utility, supported by rigorous theoretical guarantees.

Abstract

In multiclass probabilistic prediction, Utility Calibration (UC), which focuses auditing on specified utilities, has recently received attention as a way to guarantee downstream decisions while controlling computational and sample requirements. At the same time, some multiclass problems have meaningful label hierarchies that play important roles in medicine and image classification, yet how UC evaluates utility within a hierarchy remains insufficiently understood. We show that the difference between realized utility and predicted mean utility admits an exact decomposition into a sum of contributions from the internal nodes of the label tree. This decomposition shows that positive and negative contributions from different nodes can cancel, and that even when UC is small, the utility errors remaining in parts of the hierarchy need not be small. To address this problem, we propose Hierarchical Utility Calibration (HUC), which evaluates each node contribution before summation while retaining the same target utility, subgroup, and predicted-utility interval. We further provide finite-sample evaluation over all predicted-utility intervals and propose HUC-Boost, which updates only violated internal nodes, with theoretical guarantees for both.

Problem

• Utility Calibration
• Multiclass Prediction
• Label Hierarchy
• Hierarchical Utility

Innovation

• Hierarchical Utility Calibration
• Utility Calibration
• Label Hierarchy
• HUC-Boost
• Finite-sample Evaluation