🤖 AI Summary
This study addresses fairness in toxicity detection through a multidimensional lens encompassing ranking, calibration, and rejection. It systematically evaluates the interplay between training interventions—empirical risk minimization (ERM), sample reweighting, and Group Distributionally Robust Optimization (Group DRO)—and post-processing mechanisms, including temperature scaling, confidence-based rejection, and identity-specific thresholds. The findings reveal that miscalibration constitutes a subtle yet critical form of fairness violation; current training methods reshape rather than eliminate disparities; and rejection strategies can inadvertently exacerbate inter-group inequities. Specifically, ERM achieves good global calibration but exhibits severe subgroup miscalibration, reweighting improves ranking at the cost of calibration fairness, and Group DRO eliminates calibration gaps while inducing overall miscalibration. Confidence-based rejection proves effective under ERM but fails under DRO, disproportionately disadvantaging content mentioning specific identities. The work advocates for a joint multi-axis evaluation framework to holistically assess fairness.
📝 Abstract
Fairness in toxicity classification involves three integrated axes: ranking, calibration, and abstention. Training-time interventions and post-hoc safety mechanisms cannot be evaluated independently because the former determines the efficacy of the latter. We compare Empirical Risk Minimization (ERM), instance-level reweighting, and Group DRO across these axes, combined with temperature scaling, confidence-based abstention, and per-identity threshold optimization. Evaluation uses subgroup AUC, BPSN/BNSP AUC, error gaps, and per-subgroup Expected Calibration Error (ECE) with bootstrap CIs ($n = 1000$).
We report four findings. (1) Calibration disparity is a hidden fairness violation. ERM has near-perfect aggregate calibration ($0.013$) but is significantly miscalibrated across all identity subgroups ($+0.029$ to $+0.134$). (2) Training interventions reshape rather than eliminate disparity. Reweighted ERM improves ranking (BPSN AUC $+0.06$ to $+0.12$) but worsens the calibration-fairness gap by up to $+0.232$. Group DRO eliminates calibration disparity but only by becoming uniformly miscalibrated globally (ECE $0.118$). (3) Post-hoc methods inherit training failure modes. Temperature scaling fails because miscalibration is non-uniform. Confidence-based abstention works under ERM but breaks under DRO, where the risk-coverage curve rises with deferral. (4) Abstention itself is unfair. Confidence-based deferral helps background content far more than identity-mentioning content. We argue that SRAI fairness requires a multi-axis framework: methods that differ only in aggregate ranking can differ sharply in failure modes that determine real-world harm.