Robust Confidence Intervals for a Binomial Proportion: Local Optimality and Adaptivity

2025-09-05

AI Summary

This paper addresses adaptive interval estimation for binomial proportions under Huber contamination, where the contamination proportion is unknown and the adaptation cost must be characterized within a local minimax framework. We propose a novel construction method integrating robust hypothesis testing with a new monotonicity-enforcing step. Our approach is the first to precisely quantify the optimal rate degradation induced by unknown contamination, while guaranteeing exact coverage, delivering uniformly consistent confidence bounds, and maintaining computational efficiency. The method naturally extends to Poisson contamination models, preserving optimality. Furthermore, we establish a fundamental impossibility result: full adaptivity to the contamination proportion is theoretically infeasible in the node-contamination setting of Erdős–Rényi random graphs. Collectively, our results unify the theoretical limits of adaptivity and the constructive paradigms for robust interval estimation.

Technology Category

• Reasoning under Uncertainty: Stochastic Optimization
• Intelligent Robots: State Estimation
• Machine Learning: Calibration & Uncertainty Quantification

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Security and Privacy: Large-scale security measurements

Abstract

This paper revisits the classical problem of interval estimation of a binomial proportion under Huber contamination. Our main result derives the rate of optimal interval length when the contamination proportion is unknown under a local minimax framework, where the performance of an interval is evaluated at each point in the parameter space. By comparing the rate with the optimal length of a confidence interval that is allowed to use the knowledge of contamination proportion, we characterize the exact adaptation cost due to the ignorance of data quality. Our construction of the confidence interval to achieve local length optimality builds on robust hypothesis testing with a new monotonization step, which guarantees valid coverage, boundary-respecting intervals, and an efficient algorithm for computing the endpoints. The general strategy of interval construction can be applied beyond the binomial setting, and leads to optimal interval estimation for Poisson data with contamination as well. We also investigate a closely related Erdős--Rényi model with node contamination. Though its optimal rate of parameter estimation agrees with that of the binomial setting, we show that adaptation to unknown contamination proportion is provably impossible for interval estimation in that setting.

Problem

• Deriving optimal confidence intervals for binomial proportion under unknown contamination
• Characterizing adaptation cost due to ignorance of data contamination level
• Investigating interval estimation impossibility in Erdős-Rényi model with contamination

Innovation

• Robust hypothesis testing with monotonization
• Local minimax framework for unknown contamination
• Efficient algorithm for boundary-respecting intervals