Innovation
Methods, ideas, or system contributions that make the work stand out.
Legibility Trap
Participatory AI Governance
Critical Discourse Analysis
Algorithmic Harm
Community Subjectivity
Govern, Map, Measure, Absorb: The Legibility Trap in Public Sector Participatory AI Governance
🤖 AI Summary
This study addresses the tendency of community participation in public-sector AI governance to remain merely performative, failing to produce substantive systemic change. Employing critical discourse analysis on official and community documents from three public-sector AI deployments, this work draws on James C. Scott’s theoretical framework to propose the concept of the “legibility trap.” It demonstrates how institutional documentation mechanisms systematically filter out dissent that constrains state power, thereby transforming non-responsiveness into harm against community agency. Furthermore, the research identifies structural conditions under which this trap may be partially circumvented and highlights that prevailing AI ethics frameworks remain inadequate in recognizing such harms. Ultimately, this study offers a novel theoretical lens for understanding the failures of participatory governance in public-sector AI systems.
📝 Abstract
Public sector agencies have adopted participatory design as a prominent procedural safeguard proposed against algorithmic harm. Across methods, institutional contexts, and well-intentioned implementations, the participation gap persists. Communities engage, input is documented, and systems are deployed largely unchanged. This paper locates the persistence of that gap in the mechanism institutions use to document participation itself. We develop the concept of the legibility trap, in which institutional mechanisms for documenting participation systematically select for forms of community input that fit institutional categories and filter out the forms that would most constrain state power, including foundational dissent, demands for non-deployment, and what Kelly Oliver calls witnessing to structural conditions of harm. Drawing on James Scott's theory of state simplification, we demonstrate the legibility trap through critical discourse analysis of twenty-two official, community-produced, and public-record documents across three public sector AI deployments: the Allegheny Family Screening Tool, Detroit Project Green Light, and the Community Control Over Police Surveillance (CCOPS) ordinances in Oakland. We identify the structural conditions under which participation partially escapes the trap, and we theorize that repeated non-response constitutes a harm to community subjectivity that AI ethics frameworks have not yet named.
🔎 Similar Papers
No similar papers found.
