Why Sampling Is Not Choosing: Intentionality, Agency, and Moral Responsibility in Large Language Models

2026-06-11

AI Summary

This study examines whether large language models (LLMs) qualify as moral agents by investigating whether they possess the intrinsic intentionality and agency requisite for moral responsibility. Drawing on philosophical frameworks including the intentional stance, functionalism, and compatibilism, the paper rigorously distinguishes derived intentionality from intrinsic intentionality, arguing that LLM outputs amount merely to data-driven probabilistic mappings devoid of self-ascribed commitments or genuine choice. The work contends that moral responsibility presupposes a form of agency capable of sustaining commitments, and systematically refutes prevailing claims that LLMs constitute moral agents, thereby establishing that current LLMs fundamentally lack the necessary agential capacities.

Abstract

Recent advances in large language models (LLMs) have prompted claims that such systems exhibit agency or qualify as moral agents. This paper argues that these attributions are misguided. We maintain that moral responsibility requires commitment-bearing agency grounded in intrinsic intentionality and self-attributed action, and that such agency constitutes the form of free will relevant to responsibility. Although LLMs generate coherent and normatively evaluable outputs, their operation is fully characterized by probabilistic input-output mappings learned from data. Their apparent intentionality is derived rather than intrinsic, and their outputs are neither owned as commitments nor guided by reasons. Variability introduced by stochastic sampling does not amount to choice or authorship. We address objections from the intentional stance, functionalism, compatibilism, and the presence of moral reasoning in model outputs, arguing that none suffice to establish genuine agency.

Problem

• moral responsibility
• agency
• intentionality
• large language models
• free will

Innovation

• intentionality
• moral agency
• stochastic sampling
• commitment-bearing agency
• large language models