An analysis of binary isotonic regression: degrees of freedom and implications for calibration

2026-07-29

AI Summary

This work investigates the worst-case degrees of freedom of isotonic regression with binary responses and its theoretical guarantees for probability calibration. By analyzing the number of distinct fitted values and leveraging tools from analytic number theory and statistical learning theory, the authors establish—for the first time—a tight asymptotic upper bound on the degrees of freedom, with leading term $\frac{3}{(4\pi^2)^{1/3}} n^{2/3}$. Building on this result, they derive the first nontrivial upper bound on the expected calibration error (ECE) that requires no assumptions on the underlying model or data distribution, other than the response variable $Y$ being binary ($Y \in \{0,1\}$). This provides rigorous theoretical support for the use of isotonic regression in calibration tasks.

Abstract

Isotonic regression is a canonical tool for estimating monotone functions and calibrating probabilistic predictors. We provide a fully sharp finite-sample characterization of its worst-case degrees of freedom on binary samples. Specifically, we identify the binary sequences that maximize the number of distinct fitted values produced by isotonic regression. We develop a sharp bound on the degrees of freedom with a leading term of $\frac{3}{(4π^2)^{1/3}} n^{2/3}$ using analytic number theory, improving on previous bounds. We then apply this result to calibration. Calibration is a central requirement for probabilistic prediction, and isotonic regression is a widely used post-processing method for improving calibration. Building on deterministic degrees-of-freedom bounds, we derive, to our knowledge, the first nontrivial distribution-free guarantee on the Expected Calibration Error (ECE) of isotonic regression. This ECE bound is fully model-free and distribution-free, only assuming $Y \in \{0,1\}$.

Problem

• isotonic regression
• degrees of freedom
• calibration
• Expected Calibration Error
• binary samples

Innovation

• isotonic regression
• degrees of freedom
• Expected Calibration Error
• distribution-free guarantee
• binary samples