Towards User-Focused Cross-Domain Testing: Disentangling Accessibility, Usability, and Fairness

2025-01-11

AI Summary

In software testing, the user-centered concepts of fairness, usability, and accessibility suffer from conceptual conflation, ambiguous boundaries, and fragmented practice. Method: We conducted a three-tier systematic literature review (SLR) synthesizing findings from 12 recent SLRs (2014–2024) to develop the first integrated, user-focused testing framework unifying these three dimensions. Through conceptual mapping, cross-domain comparison, and meta-analysis, we clarified their theoretical distinctions and elucidated their interdependent implementation mechanisms in AI systems. Contribution/Results: We propose novel cross-cutting evaluation dimensions and actionable, integrated testing guidelines. Our key contribution is the first domain-agnostic, synergistic testing model—moving beyond single-dimension paradigms—and an operational governance pathway with concrete implementation recommendations for real-world adoption.

Technology Category

• Humans and AI: User Experience and Usability
• Philosophy and Ethics of AI: Safety, Robustness & Trustworthiness
• Cognitive Modeling & Cognitive Systems: Agent Architectures

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Responsible Web: Consent frameworks and practices on the web

Abstract

Fairness testing is increasingly recognized as fundamental in software engineering, especially in the domain of data-driven systems powered by artificial intelligence. However, its practical integration into software development may pose challenges, given its overlapping boundaries with usability and accessibility testing. In this tertiary study, we explore these complexities using insights from 12 systematic reviews published in the past decade, shedding light on the nuanced interactions among fairness, usability, and accessibility testing and how they intersect within contemporary software development practices.

Problem

• Software Development
• Fairness Testing
• Usability and Accessibility Testing

Innovation

• Software Fairness
• Usability and Accessibility Testing
• AI Data Systems