🤖 AI Summary
This work addresses the fundamental tension between fairness and economic efficiency in automated decision-making. Methodologically, it introduces the NP-EO paradigm—the first framework to rigorously incorporate Equal Opportunity (EO) constraints into the Neyman–Pearson (NP) classification framework. We formulate a joint optimization problem, derive the oracle classifier satisfying both EO and NP constraints, and design a finite-sample algorithm that, with high probability, guarantees group-level fairness (strict EO compliance) and controlled Type I error rate. Theoretically and empirically—across synthetic and real-world datasets—the algorithm achieves statistical validity (precise error control) and social validity (substantial improvement in true positive rates for disadvantaged groups), while incurring only bounded efficiency loss. The core contribution is the first exact integration of NP decision theory with group-fairness constraints, yielding a verifiable, deployable solution for high-stakes classification tasks.
📝 Abstract
Organizations often rely on statistical algorithms to make socially and economically impactful decisions. We must address the fairness issues in these important automated decisions. On the other hand, economic efficiency remains instrumental in organizations' survival and success. Therefore, a proper dual focus on fairness and efficiency is essential in promoting fairness in real-world data science solutions. Among the first efforts towards this dual focus, we incorporate the equal opportunity (EO) constraint into the Neyman-Pearson (NP) classification paradigm. Under this new NP-EO framework, we (a) derive the oracle classifier, (b) propose finite-sample based classifiers that satisfy population-level fairness and efficiency constraints with high probability, and (c) demonstrate statistical and social effectiveness of our algorithms on simulated and real datasets.