Approximating Discrimination Within Models When Faced With Several Non-Binary Sensitive Attributes

2024-08-12

arXiv.org

AI Summary

Existing fairness metrics struggle to assess discrimination under multi-valued or non-binary sensitive attributes, particularly when implicit bias arises from complex combinations of multiple attributes. Method: This paper proposes HFM (Hierarchical Fairness Metric), a fine-grained fairness measure grounded in manifold distance, which— for the first time—unifies the geometric modeling of implicit discrimination across multi-attribute configurations via manifold geometry. To ensure scalability, we design two theoretically grounded set-distance approximation algorithms: ApproxDist (with provable error bounds) and ExtendDist, enabling efficient fairness evaluation for both single-attribute multi-value and multi-attribute joint settings. Results: Experiments demonstrate HFM’s strong discriminative power; ApproxDist and ExtendDist maintain >90% accuracy while achieving 3–5× speedup on multiple benchmark datasets, significantly outperforming existing binary- or single-attribute fairness metrics. This work overcomes key modeling and computational bottlenecks in fairness quantification, establishing a scalable, interpretable, and geometrically principled framework for fairness assessment under complex sensitive attribute structures.

Technology Category

Machine Learning: Hardware-aware MLComputer Vision: Bias, Fairness & PrivacyHumans and AI: Other Foundations of Human Computation & AI

Application Category

User Modeling, Personalization and Recommendation: Fairness-aware retrieval and rankingEconomics, Online Markets and Human Computation: Fairness, privacy, and diversity in economic environmentsSearch and Retrieval-Augmented AI: Web evaluation methodologies and metrics

Abstract

Discrimination mitigation within machine learning (ML) models could be complicated because multiple factors may be interwoven hierarchically and historically. Yet few existing fairness measures can capture the discrimination level within ML models in the face of multiple sensitive attributes (SAs). To bridge this gap, we propose a fairness measure based on distances between sets from a manifold perspective, named as 'Harmonic Fairness measure via Manifolds (HFM)' with two optional versions, which can deal with a fine-grained discrimination evaluation for several SAs of multiple values. Because directly computing HFM may be costly, to accelerate its subprocedure -- the computation of distances of sets, we further propose two approximation algorithms named 'Approximation of distance between sets for one sensitive attribute with multiple values (ApproxDist)' and 'Approximation of extended distance between sets for several sensitive attributes with multiple values (ExtendDist)' to respectively resolve bias evaluation of one single SA with multiple values and that of several SAs with multiple values. Moreover, we provide an algorithmic effectiveness analysis for ApproxDist under certain assumptions to explain how well it could work. The empirical results demonstrate that our proposed fairness measure HFM is valid and approximation algorithms (i.e. ApproxDist and ExtendDist) are effective and efficient.

Problem

• Measuring discrimination in ML models with multiple non-binary sensitive attributes
• Proposing a fairness measure (HFM) for fine-grained evaluation of multiple SAs
• Developing efficient approximation algorithms (ApproxDist, ExtendDist) for bias evaluation

Innovation

• Proposes Harmonic Fairness measure via Manifolds (HFM)
• Introduces ApproxDist for single sensitive attribute
• Develops ExtendDist for multiple sensitive attributes