Fairness Research For Machine Learning Should Integrate Societal Considerations

2025-06-14

AI Summary

This paper identifies a critical gap in contemporary machine learning fairness research: its widespread neglect of institutional, historical, and political structures—leading to fairness metrics that are decontextualized from real-world social dynamics. To address this, we introduce the “societally embedded fairness” framework, which integrates computational social science, critical algorithm studies, and formal modeling to systematically expose how human-AI feedback loops exponentially amplify minor societal biases. Moving beyond purely technical approaches—such as statistical parity or adversarial debiasing—the framework shifts fairness research from model-level tool optimization toward governance of sociotechnical systems. Key contributions include: (1) establishing socially sensitive evaluation principles; and (2) providing actionable, interdisciplinary guidelines for policymakers and engineers. Collectively, these advances substantively challenge and extend the dominant technocratic fairness paradigm.

Technology Category

Machine Learning: Ethics, Bias, and FairnessPhilosophy and Ethics of AI: Bias, Fairness & EquityHumans and AI: Other Foundations of Human Computation & AI

Application Category

Social Networks and Social Media: Fairness and bias in social network and social media analysisEconomics, Online Markets and Human Computation: Fairness and ethical considerations in crowd work and in human-in-the-loop AI systemsUser Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Enhancing fairness in machine learning (ML) systems is increasingly important nowadays. While current research focuses on assistant tools for ML pipelines to promote fairness within them, we argue that: 1) The significance of properly defined fairness measures remains underestimated; and 2) Fairness research in ML should integrate societal considerations. The reasons include that detecting discrimination is critical due to the widespread deployment of ML systems and that human-AI feedback loops amplify biases, even when only small social and political biases persist.

Problem

• Underestimated importance of proper fairness measures in ML
• Need to integrate societal considerations in ML fairness research
• Human-AI feedback loops amplify biases despite small societal biases

Innovation

• Integrate societal considerations in ML fairness
• Focus on properly defined fairness measures
• Address human-AI feedback loops amplifying biases