Fairness Auditing: Lower Bounds on Company Manipulation

2026-08-01

AI Summary

This study addresses the vulnerability of fairness auditing in resource-constrained settings, where computationally unbounded firms may manipulate their models post-audit to circumvent fairness constraints, thereby undermining fairness guarantees in high-stakes applications. The problem is formalized as a minimax optimization between a resource-limited auditor and a strategic firm, yielding the first quantification of the fundamental lower bound on post-audit manipulation under a finite auditing budget. The theoretical analysis integrates group imbalance and tolerance parameters, deriving a worst-case lower bound on demographic parity violation that jointly depends on the audit budget, degree of group imbalance, and allowable tolerance. Empirical evaluations on both linear models and neural networks confirm that while increased auditing resources can mitigate manipulation, they cannot fully eliminate the strategic firm’s ability to exploit residual gaps.

Abstract

Fairness audits are increasingly mandated in high-stakes applications such as hiring, lending, and automated decision-making. Recent work has established fundamental impossibility results for black-box fairness auditing, showing that sufficiently expressive models can evade any auditing strategy. We complement these results by quantifying the extent of unavoidable post-audit manipulation under finite audit resources. We formulate fairness auditing as a min-max optimization between a computationally unbounded company and a budget-constrained auditor. We study two auditing regimes: (i) a budgeted auditor that certifies fairness using a fixed-size audit set, and (ii) a budgeted α-tolerant auditor that additionally requires the audit set to estimate the fairness of the certified model within an α approximation. For both settings, we derive explicit lower bounds on the worst-case post-audit demographic parity deviation as functions of the audit budget, group imbalance, and fairness tolerance. Finally, we empirically illustrate these theoretical limits using simple audit-set construction heuristics with linear and neural network classifiers. Our results demonstrate that increasing audit resources reduces, but does not eliminate, the scope for post-audit manipulation, highlighting fundamental limitations of finite-budget fairness certification.

Problem

• fairness auditing
• post-audit manipulation
• audit budget
• demographic parity
• impossibility results

Innovation

• fairness auditing
• min-max optimization
• demographic parity
• audit budget
• post-audit manipulation