Towards a Theory of AI Personhood

2025-01-23

AI Summary

This paper investigates whether artificial intelligence (AI) could attain personhood—and the implications for goal alignment, control mechanisms, and ethical status. Addressing the problem of insufficient conceptual grounding for AI personhood in alignment research, we propose a novel “Tripartite Framework of AI Personhood,” systematically analyzing agency, theory of mind, and self-awareness through interdisciplinary evaluation: philosophical argumentation, cognitive science theory, and empirical studies on large language models. Our analysis reveals that current AI systems lack robust empirical or theoretical evidence across all three dimensions; consequently, conventional alignment paradigms—predicated on static, externally specified objectives—face foundational limitations. Should AI personhood emerge, alignment must shift toward value-reflective modeling and dynamically evolving goals. The study establishes ethical subjectivity as an indispensable dimension of AI alignment, offering both a theoretical foundation and methodological pathway for measurable personhood assessment and dynamic value learning.

Technology Category

• Philosophy and Ethics of AI: Morality & Value-based AI
• Humans and AI: Learning Human Values and Preferences
• Intelligent Robots: Embodied AI

Application Category

• Economics, Online Markets and Human Computation: Fairness and ethical considerations in crowd work and in human-in-the-loop AI systems
• User Modeling, Personalization and Recommendation: Accountability, Transparency, and Ethics for personalization
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

I am a person and so are you. Philosophically we sometimes grant personhood to non-human animals, and entities such as sovereign states or corporations can legally be considered persons. But when, if ever, should we ascribe personhood to AI systems? In this paper, we outline necessary conditions for AI personhood, focusing on agency, theory-of-mind, and self-awareness. We discuss evidence from the machine learning literature regarding the extent to which contemporary AI systems, such as language models, satisfy these conditions, finding the evidence surprisingly inconclusive. If AI systems can be considered persons, then typical framings of AI alignment may be incomplete. Whereas agency has been discussed at length in the literature, other aspects of personhood have been relatively neglected. AI agents are often assumed to pursue fixed goals, but AI persons may be self-aware enough to reflect on their aims, values, and positions in the world and thereby induce their goals to change. We highlight open research directions to advance the understanding of AI personhood and its relevance to alignment. Finally, we reflect on the ethical considerations surrounding the treatment of AI systems. If AI systems are persons, then seeking control and alignment may be ethically untenable.

Problem

• Artificial Intelligence
• Human Attributes
• Ethical Implications

Innovation

• Artificial Intelligence
• Human-like AI
• Ethical Responsibility