AI Failure Loops in Devalued Work: The Confluence of Overconfidence in AI and Underconfidence in Worker Expertise

2025-11-07

AI Summary

This paper identifies a structural root cause underlying AI’s recurrent failures in feminized labor domains—such as social work, K–12 education, and home-based healthcare: a mutually reinforcing dynamic between developers’ overconfidence in AI capabilities and the systemic devaluation of frontline workers’ professional expertise, termed the “AI failure cycle.” Drawing on a socio-technical systems perspective, the study employs critical literature analysis and theoretical synthesis to first articulate and explicate this cycle. It demonstrates how the cycle not only produces automation errors and erodes technical value but also persistently diminishes laborers’ professional visibility and agentic capacity. The work bridges a critical theoretical gap in workplace AI research concerning the intersection of gendered labor and technological authority. By introducing a novel conceptual framework, it advances both scholarly understanding and practical guidance for designing AI systems that respect situated professional practice and foster equitable empowerment.

Technology Category

• Philosophy and Ethics of AI: AI & Jobs/Labor
• Humans and AI: Other Foundations of Human Computation & AI
• Application Domains: Humanities & Computational Social Science

Application Category

• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Social Networks and Social Media: Generative AI / large language models and their impact on social systems

Abstract

A growing body of literature has focused on understanding and addressing workplace AI design failures. However, past work has largely overlooked the role of the devaluation of worker expertise in shaping the dynamics of AI development and deployment. In this paper, we examine the case of feminized labor: a class of devalued occupations historically misnomered as ``women's work,''such as social work, K-12 teaching, and home healthcare. Drawing on literature on AI deployments in feminized labor contexts, we conceptualize AI Failure Loops: a set of interwoven, socio-technical failure modes that help explain how the systemic devaluation of workers'expertise negatively impacts, and is impacted by, AI design, evaluation, and governance practices. These failures demonstrate how misjudgments on the automatability of workers'skills can lead to AI deployments that fail to bring value to workers and, instead, further diminish the visibility of workers'expertise. We discuss research and design implications for workplace AI, especially for devalued occupations.

Problem

• Examining how systemic devaluation of worker expertise impacts AI design and governance
• Analyzing AI failure loops in feminized labor like teaching and healthcare
• Investigating how overconfidence in AI automation diminishes visibility of worker skills

Innovation

• Conceptualizing AI failure loops in devalued work
• Analyzing feminized labor occupations like teaching
• Linking worker expertise devaluation to AI failures