Practitioner Insights on Fairness Requirements in the AI Development Life Cycle: An Interview Study

2025-12-15

AI Summary

This study addresses the challenge of operationalizing fairness requirements in the AI software development lifecycle (SDLC): although practitioners widely acknowledge AI fairness as critical, it is routinely deprioritized amid functional delivery pressures and tight deadlines—exposing three core bottlenecks: ambiguous definitions, absent metrics, and missing process integration. Through cross-cultural, semi-structured interviews with 26 AI practitioners across 26 countries and thematic qualitative analysis, we systematically identify real-world fairness gaps across SDLC phases—requirements elicitation, modeling, verification, and trade-off negotiation—from a software engineering perspective. Our key contribution is a framework advocating co-defined, context-sensitive fairness metrics involving diverse stakeholders, and their formal integration into SDLC artifacts and workflows—thereby establishing a foundation for auditable, evolvable, fairness-aware AI engineering practice.

Technology Category

Philosophy and Ethics of AI: Bias, Fairness & EquityMachine Learning: Ethics, Bias, and FairnessNatural Language Processing: Ethics — Bias, Fairness, Transparency & Privacy

Application Category

Economics, Online Markets and Human Computation: Fairness and ethical considerations in crowd work and in human-in-the-loop AI systemsUser Modeling, Personalization and Recommendation: Fairness-aware retrieval and rankingSocial Networks and Social Media: Fairness and bias in social network and social media analysis

Abstract

Nowadays, Artificial Intelligence (AI), particularly Machine Learning (ML) and Large Language Models (LLMs), is widely applied across various contexts. However, the corresponding models often operate as black boxes, leading them to unintentionally act unfairly towards different demographic groups. This has led to a growing focus on fairness in AI software recently, alongside the traditional focus on the effectiveness of AI models. Through 26 semi-structured interviews with practitioners from different application domains and with varied backgrounds across 23 countries, we conducted research on fairness requirements in AI from software engineering perspective. Our study assesses the participants' awareness of fairness in AI / ML software and its application within the Software Development Life Cycle (SDLC), from translating fairness concerns into requirements to assessing their arising early in the SDLC. It also examines fairness through the key assessment dimensions of implementation, validation, evaluation, and how it is balanced with trade-offs involving other priorities, such as addressing all the software functionalities and meeting critical delivery deadlines. Findings of our thematic qualitative analysis show that while our participants recognize the aforementioned AI fairness dimensions, practices are inconsistent, and fairness is often deprioritized with noticeable knowledge gaps. This highlights the need for agreement with relevant stakeholders on well-defined, contextually appropriate fairness definitions, the corresponding evaluation metrics, and formalized processes to better integrate fairness into AI/ML projects.

Problem

• Investigates fairness awareness and practices in AI development life cycles
• Examines how fairness is integrated and balanced with other software priorities
• Identifies gaps in consistent fairness implementation across diverse AI projects

Innovation

• Interview practitioners on fairness in AI development
• Assess fairness awareness across software development lifecycle
• Identify gaps and need for formalized fairness processes