The Method of Critical AI Studies, A Propaedeutic

2024-11-28

arXiv.org

AI Summary

This paper identifies and critiques three methodological biases in critical AI studies: benchmark-case reductionism (overreliance on canonical examples), black-box analogism (uncritical adoption of outdated computational metaphors), and stack-based causalism (linear, deterministic attribution of algorithmic harms). To address these interlocking limitations, the paper introduces—systematically for the first time—the “introductory” methodological framework, centered on humanistic close reading. Integrating cultural text analysis, conceptual history, and critical discourse analysis, this approach transcends the binary constraints of technological determinism and purely speculative theorization. Emphasizing contextual sensitivity and epistemic reflexivity, the framework advances methodological self-awareness for AI ethics, policy analysis, and cultural inquiry, while offering a pedagogically robust, interdisciplinary tool for teaching and research.

Technology Category

• Philosophy and Ethics of AI: Bias, Fairness & Equity
• Machine Learning: Ethics, Bias, and Fairness
• Humans and AI: Understanding People, Theories, Concepts and Methods

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Social Networks and Social Media: Fairness and bias in social network and social media analysis
• Economics, Online Markets and Human Computation: Fairness and ethical considerations in crowd work and in human-in-the-loop AI systems

Abstract

We outline some common methodological issues in the field of critical AI studies, including a tendency to overestimate the explanatory power of individual samples (the benchmark casuistry), a dependency on theoretical frameworks derived from earlier conceptualizations of computation (the black box casuistry), and a preoccupation with a cause-and-effect model of algorithmic harm (the stack casuistry). In the face of these issues, we call for, and point towards, a future set of methodologies that might take into account existing strengths in the humanistic close analysis of cultural objects.

Problem

• Overestimating explanatory power of individual samples
• Dependency on outdated computational theoretical frameworks
• Focus on cause-and-effect model of algorithmic harm

Innovation

• Addresses methodological issues in critical AI studies
• Proposes humanistic close analysis of cultural objects
• Critiques overreliance on theoretical computational frameworks