Nonparametric Uniform Inference in Binary Classification and Policy Values

2025-11-18

AI Summary

In cost-sensitive binary classification and policy value evaluation, nonparametric inference faces challenges including slow convergence, nonstandard limiting distributions, and failure of point identification. To address these, this paper proposes a nonparametric uniformly consistent inference method based on a strictly convex surrogate loss function. The method achieves, for the first time in a nonparametric setting, point identification of the optimal classification policy function while attaining the standard nonparametric convergence rate. Moreover, the estimated optimal policy value is shown to be √n-asymptotically normal, enabling Gaussian approximation and valid statistical inference. Theoretical analysis integrates empirical process theory, and simulation studies alongside an empirical application to the Job Training Partnership Act (JTPA) dataset confirm estimation consistency, inferential validity, and reliable quantification of policy welfare effects.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Search and Optimization: Non-convex Optimization
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• User Modeling, Personalization and Recommendation: Metrics for user behavior and evaluating success
• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

We develop methods for nonparametric uniform inference in cost-sensitive binary classification, a framework that encompasses maximum score estimation, predicting utility maximizing actions, and policy learning. These problems are well known for slow convergence rates and non-standard limiting behavior, even under point identified parametric frameworks. In nonparametric settings, they may further suffer from failures of identification. To address these challenges, we introduce a strictly convex surrogate loss that point-identifies a representative nonparametric policy function. We then estimate this surrogate policy to conduct inference on both the optimal classification policy and the optimal policy value. This approach enables Gaussian inference, substantially simplifying empirical implementation relative to working directly with the original classification problem. In particular, we establish root-$n$ asymptotic normality for the optimal policy value and derive a Gaussian approximation for the optimal classification policy at the standard nonparametric rate. Extensive simulation studies corroborate the theoretical findings. We apply our method to the National JTPA Study to conduct inference on the optimal treatment assignment policy and its associated welfare.

Problem

• Addresses slow convergence in nonparametric binary classification problems
• Solves identification failures in cost-sensitive policy learning frameworks
• Enables Gaussian inference for optimal policy values and assignments

Innovation

• Introduces strictly convex surrogate loss for identification
• Enables Gaussian inference for optimal policy value
• Derives Gaussian approximation for classification policy